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ABSTRACT 


The  first  problem  in  the  theory  of  signal  detectability  deals 
with  the  decision  between  two  alternative  responses,  corresponding 
to  two  possible  classes  of  causes  of  an  observation.  When  the  goal 
of  a  decision  process  is  to  achieve  the  highest  quality  of  terminal 
decision,  the  Receiver  Operating  Characteristic  curve  (ROC  curve) 
contains  all  of  the  information  necessary  for  the  evaluation  of  the 
decision  process.  This  present  work  introduces  the  ROC  charac¬ 
ter,  which  is  isomorphic  to  the  ROC  curve. 

The  formal  development  is  based  on  two  key  facts.  The  first 
is  the  fundamental  theorem:  if  £(X)  is  the  likelihood  ratio  of  an 
observation,  then  the  likelihood  ratio  oi  £  is  £  itself.  The  second 
is  the  main  theorem  on  ROC  characters:  each  ROC  character  is  iso¬ 
morphic  to  a  univariate  probability  distribution  that  possesses  a 
moment  generating  function.  The  character  convolution  theorem 
and  the  character  addition  theorem  follow  directly  from  these. 

Families  of  ROC  curves  are  developed  from  the  main  theorem 
on  ROC  characters.  The  normal,  binormal,  Q-table,  power,  and 
several  discrete  families  of  ROC  curve  have  appeared  in  the  litera¬ 
ture.  The  new  families  include  the  Pearson  type  HI,  Fisher- Tippett 
doubly  exponential,  H-type,  Poisson,  and  the  regular  conics. 


Additional  families  are  generated  from  these  by  use  of  the  meta¬ 
static  relation,  and  the  convolution  and  addition  theorems. 

ROC  curves  contain  information  about  the  performance 
of  other  two-cause  decisions  besides  the  two- response  decision. 
Several  are  considered  that  are  used  in  the  testing  of  human  per¬ 
ception;  namely,  the  symmetric  forced  choice  decision,  type  II 
decisions,  the  rating  scale  procedure,  and  the  analysis  of  a  de¬ 
cision  based  on  the  reports  of  multiple  observers. 


FOREWORD 


This  report  contains  foundational  work  in  signal  detection 
theory.  The  primary  purpose  of  the  work  was  to  provide  a  tech¬ 
nique  for  cataloging  performance  curves  of  detection-decision  de¬ 
vices,  and  to  expound  upon  properties  of  specific  performance 
curves.  Its  usefulness  lies  in  the  exhaustive  detailing,  which  the 
author  hopes  will  provide  detection  theory  researchers  with  both 
insight  and  detailed  information  about  performance  curves  they 


encounter. 
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CHAPTER  I 


INTRODUCTION 


The  theory  of  signal  detectability  has  grown  up  in 
an  interdisciplinary  way, combining  parts  of  the  traditional  fields  of 
mathematics,  pnysics,  engineering,  physiology,  and  psychology. 

The  skeleton  on  which  the  theory  is  built  is  the  mathematics  of 
decision  theory.  The  analytic  description  of  the  observation  and  per¬ 
ception.  tne  interferences  and  the  cause- effect  relationships  are  drawn 
chiefly  from  physics  and  physiology.  The  subsequent  realization  of  deci¬ 
sion  mechanisms  is  part  of  engineering  when  the  realization  is  in  hard¬ 
ware,  part  of  psychology  when  one  is  dealing  with  a  human  or  animal 
decision  mechanism,  and  falls  to  other  specialties  when  'he  realization 
is  a  computer  program  or  a  finite  automaton.  The  evaluation  of  the  perfor¬ 
mance  of  the  decision  mechainsm  has  similarly  fallen  to  various  fields, 

depending  upon  the  application. 

This  present  work  concentrates  on  models  for  the  third 

aspect,  evaluation  of  detectability.  Consideration  is  restricted  to  those 
situations  where  decision  quality  is  the  sole  goal  of  the  decision  device, 
and  the  sole  consideration  in  the  evaluation.  In  the  past,  evaluation 
models  have  come  principally  from  problems  in  electrical  engineering 
Yet  the  theory  is  applicable  to  any  information  process  involving  both 
observation  and  subsequent  (binary)  decision.  Today's  applications 
involve  such  diverse  areas  as  radar,  sonar,  and  digital  communications; 
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the  psychophysics  of  human  and  animal  perception;  information  retrieval 
and  medical  diagnoses  by  machines.  One  major  pu.  pose  of  this  present 
work  is  to  furnish  a  variety  of  evaluation  models.  A  second  major 
purpose  is  to  develop  sufficient  mathematical  structure  to  form  a 
basis  for  classification  of  these  models,  and  to  form  a  basis  for  the 
generation  of  additional  models.  It  is  hoped  that  this  work  will  be  useful 
to  those  making  mathematical  analyses  of  detection  devices  by  specifying 
the  parameters  that  must  be  determined.  It  is  hoped  that  it  will  be 
useful  to  those  experimenters  with  performance  data,  that  it  will  help 
classify  the  data  and  indicate  appropriate  models.  For  the  mathematically 
inclined  tnis  paper  develops  a  special  T,_  function  called  the  ROC  charac- 
ter.  that  is  one-to-one  with  the  graphical  presentation,  the  ROC  curve. 

I  hope  those  in  communications  sciences  with  a  broad  interest  in  observa¬ 
tion-decision  procedures  will  be  able  to  learn  what  information  an  ROC 
curve  furnishes,  and  what  information  it  omits. 

We  shall  concentrate  on  the  simplest  of  all  observation- 
decision  problems  --  the  two-by-two  problem.  This  work  is  directly 
applicable  to  the  two-by-E*  (rating)  problem;  however,  no  generalization 
to  finite  higher  order  decision  problems  will  be  made.  In  the  two-by-two 
problem,  it  is  assumed  that  two  alternative  decisions  may  be  made, 
corresponding  to  two  mutually  exhaustive  and  exclusive  classes  of  causes. 
In  order  to  make  any  sensible  decision  a  given  cause  (such  as  a  specific 
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and  not  transfer  back  and 


signal)  must  belong  to  one  class  or  the  other 


forth.  Tnis  is  the  reason  for  the  word  "exclusive.  "  The  assumption 


"exhaustive"  means  that  no  other  cause  than  those  listed  in  the  two  classes 


can  occur. 

This  present  work  will  deal  principally  with  three  subjects: 
(1)  the  ROC  curve,  (2)  the  likelihood  ratio  and  related  functions,  and 
(3)  a  new  function,  ir(z)  ,  called  the  ROC  character.  In  this  first 
chapter  the  basic  relations  between  the  ROC  curve  and  the  likelihood 
ratio  will  be  reviewed  and  the  proof  given  for  the  fundamental  theorem: 

The  likelihood  ratio  of  the  likelihood  ratio  is  the  likelihood  ratio.  " 

This  theorem  has  been  known  and  used  by  several  specialists  in  the 
Held,  but  apparently  has  not  previously  been  rigorously  established. 

In  Chapter  II  the  ROC  character  will  be  formally  introduced  and  it  will 
be  shown  that  it  contains  information  that  is  unavailable  in  the  likelihood 
ratio  itself.  Even  further  it  will  be  shown  that  the  likelihood  ratio  and 
tne  ROC  character  contain  the  two  complementary  and  separate  aspects 
of  detection;  namely,  that  likelihood  ratio  is  relevant  to  the  design  of 
tf  decision  device  and  that  the  ROC  character  is  relevant  to  its  perfor¬ 
mance.  In  Chapter  III  the  ROC  character  will  be  used  to  establish  a 
basis  of  classification  of  types  of  ROC  curves.  The  primary  analytic 
result  in  Chapter  III  is  that  the  class  of  all  ROC  characters  corresponds 
to  the  class  of  probability  functions  that  are  sufficiently  smooth  and 
concentrated  to  have  moment  generating  functions.  This  correspondence 
yields  the  set  of  classical  ROC  curves. 

Most  of  the  ROC  models  available  heretofore  have  come 
from  radar  detection  problems.  In  Chapter  IV  these  few  are  collected 
and  analyzed  in  terms  of  their  ROC  character.  Chapters^.  VI  and  VII 


each  treat  at  length  one  broad  class  of  ROC.  the  algebraic,  the  truncated 
normal,  and  the  binormal.  Chapter  VIII  develops  the  Convolution 
Theorem  for  ROC  characters,  and  the  Fishsr-Tippett  ROC  character, 
both  arising  in  cases  of  multiple  observations.  Chapter  IX  presents 
the  ROC  character  Addition  Theorem,  and  two  double  parameter  charac¬ 
ters,  the  Pearson  Type  III,  and  the  H-type.  Chapter  X  is  devoted  to 
discussion  of  selected  topics  that  have  arisen  in  the  literature. 


L  1  Review  of  Detection  Theory 

The  procedure  that  will  be  followed  in  this  section  has 
two  gross  steps.  First,  the  graph  called  the  receiver  operating  charac¬ 
teristic  (ROC)  will  be  introduced  and  used  to  evaluate  detection  per¬ 
formance.  Secondly,  a  specific  function  of  observation  called  the 
"likelihood  ratio"  will  be  introduced  and  it  will  be  explained  why 
optimum  detection  is  based  on  likelihood  ratio. 

There  are  many  proofs  that  the  likelihood  ratio  for  de¬ 
tection  problems  is  optimum.  The  most  common  proofs  are  based  on 
specific  definitions  of  optimum,  such  as  "maximum  expected  value." 
"minimum  loss,”  or  "Neyman- Pearson  test.  "  The  proof  used  here 
has  been  chosen  to  emphasize  what  is  well  known,  but  really  unknown 
to  many  --  that  the  optimumness  of  likelihood  ratio  does  not  require 
that  any  specific  quantity  be  maximized  or  minimized.  All  that  is 
required  is  that  decision  quality  is  the  goal  and  that  the  definition  of 
decision  quality  reflects  a  preference  for  correct  decisions  over  mistakes. 


At  this  point,  we  must  begin  to  introduce  formal  notation  in 
order  to  be  specific,  and  to  be  able  to  manipulate  mathematically  certain 
rather  simple  quantities.  Many  types  of  notation  might  be  used,  all  of 
which  appear  ii  decision  theory  and  detection  theory.  One  might  choose 
to  emphasize  the  generality  of  the  work  by  using  abstract  notation.  I 
have  chosen  to  utilize  the  notation  common  in  radar  and  sonar  signal 
detection,  in  the  hopes  that  the  nmemonic  benefits  gained  will  over¬ 
shadow  the  apparent  loss  of  generality.  Specifically,  the  two  classes  of 
causes  will  be  signified  by  N  (noise  alone)  and  SN  (there  is  a  signal  in 
the  noise).  Two  alternative  decisions  that  the  decision  device  can  make 
are  to  sound  th«  alarm  ("A",  the  decision  that  a  signal  was  present)  or 
to  conclude  that  only  background  was  present,  the  decision  "B”.  When 
the  cause  is  noise  alone,  N,  the  decisions  correspond  to  a  false  alarm 
and  a  correct  response,  respectively;  when  the  cause  is  SN  the  decisions 
correspond  to  a  detection  and  a  miss,  respectively. 

The  formal  description  of  a  decision  device  is  a  probability 
function.  For  each  possible  observation  the  description  is  the  probability 
that  the  device  will  response  "A".  One  might  feel  that  a  description  cf 
a  decision  device  would  correspond  to  a  listing  of  those  observations  for 
which  the  decision  device  would  response  "A”.  (On  the  remainder  of  the 
observations  the  decision  device  would  respond  "B". )  Indeed,  this  has 
been  the  writer’s  standard  approach  in  electronic  problems  (Ref.  1,  p. 
174).  Although  tnis  description  is  quite  adequate  for  a  great  many  pur¬ 
poses,  it  does  not  cover  those  devices  which  may  base  their  decision  in 
part  on  extraneous  quantities:  for  example  devices  whose  decisions  are 


affected  by  internal  noise  or  devices  with  some  nonstationary  behavior 
in  the  decision  process  (such  as  a  jittering  decision  threshold.  )  The 
decision  probability  function  is  written  pO’A'Mx),  where  x  is  a  general 
notation  for  the  observation  on  which  the  device  bases  its  decision. 

In  Fig.  1. 1  the  coordinate  systems  for  the  ROC  are 
drawn  on  a  blank  unit  square.  The  horizontal  axis  indicated  on  the 

bottom  is  called  the  probability  of  the  false  alarm.  It  represents  the 
probability  of  a  mistake,  the  probability  of  making  a  decision  "A”  when 
the  correct  decision  would  have  been  ”B.  "  The  probability  of  the  corres¬ 
ponding  correct  decision  is  shown  on  the  horizontal  axis  above  the  ROC 
and  is  simply  one  minus  the  probability  of  a  false  alarm.  (The  device  is 
forced  to  make  either  an  "A"  or  "B"  decision. )  The  left  vertical  scale  is 
the  probability  of  making  the  "A"  decision  when  it  is  correct,  and 
is  called  the  probability  of  detection.  Had  the  alternative  decision 
"B"  been  made  the  action  would  have  been  labeled  a  miss  and  its 
probability  is  simply  one  minus  the  probability  of  detection.  This 
probability  is  indicated  on  the  right-hand  side  of  the  graph  on  the 
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vertical  axis.  Traditionally,  one  suppresses  the  upper  and  right- 
hand  coordinates,  listing  only  the  probabilities  of  false  alarm  and 
detection,  since  the  other  two  probabilities  can  be  obtained  readily 
from  these.  The  term  kOC  first  appeared  in  the  paper  by  Peterson, 
Birdsall,  and  Fox  (Ref.  1)  .  Curves  drawn  on  the  ROC  grid  have 
also  been  referred  to  as  contours  r"  iso-effectiveness,  or  contours 
of  equivalent  performance.  The  traditional  form  of  the  curve  that 
appears  in  statistics  and  some  work  in  signal  detection  plots  the 
probability  of  a  miss  on  the  vertical  axis  so  that  the  ROC  is  the 
mirror  image  of  the  form  used  here.  In  statistics  such  a  curve  is 
called  the  power  curve,  where  the  probability  of  false  alarm  is  called 
t'ne  critical  level  and  the  power  is  one  minus  the  probability  of  a  miss, 
that  is.  the  probability  of  detection. 

Our  formalism  for  a  decision  device  is  the  probability 
function  describing  the  probability  of  deciding  "A”  for  each 
observation  x  on  which  a  decision  is  based. 

Decision  Device:  p("A"  I  x)  (1.  i) 

Our  formalism  for  the  observation  statistics,  describing  a  specific 
problem,  are  the  measures  of  the  probability  of  occurrence  of  the 
observation,  x,  for  each  of  the  cause  classes. 


Observation  Statistics:  p(x  I  SN),  ju(xlN) 


(1.2) 


These  observation  statistics  refer  specifically  to  those  observations 
on  which  a  decision  will  be  based.  This  is  more  restrictive  than  the 
general  use  of  the  same  term  which  might  mean  the  statistics  of  the 
entire  random  process  of  which  the  observation  is  just  one  portion. 

A  specific  observation  situation  is  defined  by  specifying  the  obser¬ 
vations  on  which  a  decision  may  be  based,  and  the  observation 
statistics,  Eq.  1.2. 

The  evaluation  of  a  specific  decision  device  in  a 
specific  observation  situation  is  the  ROC  point  given  by  me  proba¬ 
bility  of  a  false  alarm  and  probability  of  a  detection. 

ROC  Evaluation  Point:  P(AlN)  =  f  p("A"|x)  dji'xlN) 

(1  3) 

P(AlSN)  =  f  p("A"lx)  dp(xlSN) 

In  Appendix  A,  taken  from  course  notes  for  the  Summer  Intensive 
Course  on  Random  Process,  1961-62-63,  it  is  shown  that  the  ROC 
plot  of  all  possible  decision  devices  in  any  specific  decision  situation 
is  convex,  contains  the  chance  diagonal,  and  is  symmetric  about  the 
center-point.  The  proofs  are  simple  but  tedious.  The  results  are 
shown  in  the  following  figures.  In  Fig.  1.2  the  chance  diagonal  has 
been  added  to  the  blank  ROC.  This  is  the  locus  of  performance  where 

i 

the  detection  probability  is  equal  to  the  false  alarm  probability.  Such 
performance  may  be  obtained  with  a  decision  device  that  reacts  indepen¬ 
dently  of  the  observation,  and  hence  is  described  as  chance.  Performance 
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falling  below  this  diagonal  is  considered  poorer  than  chance  and  per¬ 
formance  above  the  diagonal  is  considered  better  than  chance. 


Fig.  1.2.  The  "chance  diagonal  ” 


We  have  made  no  specific  definition  of  optimum.  However, 
let  us  consider  some  specific  point  below  the  chance  diagonal  and  compare 
it  to  the  point  directly  above  it  on  the  chance  diagonal.  Both  have  the 
same  performance  under  the  condition  N.  The  point  on  the  chance 
diagonal  has  a  higher  detection  probability  than  the  point  under  discussion 
below  the  chance  diagonal.  We  did  specify  that  correct  decisions  are 
to  be  preferred  to  incorrect  decision,  that  higher  correct-decision 
probabilities  and  correspondingly  lower  error  probabilities  will  be 
considered  desirable.  Therefore  the  chance  diagonal  is  better  than 
any  point  plotted  below  it. 

The  convex  closure  of  two  points  plotted  on  the  ROC 
grid  consists  of  the  straight  line  joining  the  two  points.  The  implication 
is  that  if  two  decisions  devices  are  possible  which  plot  at  points 
Wj  and  Wg.  then  it  is  known  that  a  continuum  of  decision  devices 
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exist  which  will  perform  as  indicated  by  the  line  between  the  points 


w. 


and  Wg . 


Fig.  1. 3.  The  ’’convex  closure"  of  two  points 

Consider  two  decision  devices  which  disagree  on  each 
individual  decision.  The  probability  of  "A"  of  the  second  device  will 
ue  the  probability  of  "B"  of  the  first  device. 


P("A2”lN)  =  P(’’Bj"lN) 
P(mA2"|SN)  =  P(”B1’’ISN) 


(1.4) 


Equation  (1.  4)  is  the  reason  the  ROC  region  exhibits  the  midpoint 
symmetry  that  is  indicated  geometrically  in  Fig.  1.  4. 


The  purpose  of  this  detailed  geometric  examination 
was  to  determine  the  implications  of  the  existence  of  a  single  ncn- 
chance  point.  If  there  is  a  particular  decision  device  with  nonchance 
performance,  plotted  at  Wj  ,  then  it  follows  directly  that  the  region  of 
obtainable  performance  with  the  given  observation  statistics  contains 
at  least  the  parallelogram  indicated  in  Fig.  1.  5.  The  boundary  of  this 
parallelogram  is  the  convex  closure  of  the  points  (0,  0),  w^  (1, 1), 
and  the  opposite  of  Wj.  Every  point  in  the  interior  lies  on  many 
straight  lines  between  boundary  points. 


Fig.  1.  5.  Convex  closure  of  a  point,  its  opposite,  and  chance 


The  plot  of  all  possible  ROC  performance  points  for  a 
specific  observation  situation  is  called  the  ROC  region.  The  ROC 
region  is  a  function  of  the  ob  servation  situation,  not  of  any  particular 


decision  device.  Different  observation  situations  may  lead  to  dif¬ 
ferent  ROC  regions.  If  two  observation  situations  have  the  same  ROC 
region,  they  are  said  to  be  equally  detectable  situations.  Two  observa¬ 
tion  situations  may  differ  in  all  respects;  the  observations  on  which 
decisions  are  based,  and  the  observation  statistics.  If  two  observation 
situations  have  the  same  set  of  observations,  but  differ  in  one  of  the 
measures  in  the  observation  statistics,  they  are  considered  as  different 
situations. 

For  the  remainder  of  this  section  (1.1)  we  shall  consider 
the  observation  situation  fixed,  and  the  ROC  region  known.  If  the 
ROC  region  is  as  shown  in  Fig.l.  5,  then  one  should  desire  to  operate 
somewhere  along  the  upper  boundary.  This  is  because  each  point  p 
on  the  interior  represents  a  higher  miss  probability  than  the  upper 
boundary  point  directly  above  p,  and  a  higher  false  alarm  probability 
than  the  upper  boundary  point  directly  to  the  left  of  p.  The  upper 
boundary  of  the  plot  of  the  ROC  region  is  called  the  optimum  ROC 
curve.  If  one  restricts  himself  to  only  the  upper  boundary  instead 
of  the  entire  region,he  can  still  maximize  any  specific  goal,  that  is, 
he  can  maximize  every  specific  definition  of  performance. 

The  second  step  in  this  section  is  to  establish  that 

the  ROC  region  is  closed,  meaning  that  the  upper  boundary  actually 

!  ! 

exists  and  is  obtainable,  and  that  the  widely  acclaimed  likelihood  ratio 

will  always  lead  one  to  performance  on  the  upper  boundary.  Of  special 


interest  are  those  decision  devices  which  are  based  on  the  real  quantity 
likelihood  ratio  in  a  special  way.  A  decision  is  said  to  be  based  on  a 
real  decision  axis  d(x)  and  a  cut  value  /3 ,  if  the  decision  "A"  occurs 
with  probability  one  whenever  d(x)  is  greater  than  p  and  occurs 
with  probability  zero  whenever  d(x)  is  less  than  /3.  Whenever  d(x) 
equals  ,3,  the  decision  probability  is  some  value,  r,  between  zero 
and  one. 

Because  this  present  treatment  is  slightly  more  general 
than  the  usual  likelihood  ratio  treatment,  such  as  is  given  in  Appendix  A, 
we  must  go  back  to  the  foundations  of  the  likelihood  ratio  principle. 

The  likelihood  ratio  of  an  observation  is  the  connecting  link  between 
two  possible  measures  of  occurrence  of  that  observation.  The  Radon- 
Nicodym  Theorem  (Ref.  2)  establishes  this  connecting  link.  Consider 
two  observation  statistics  /i ( x  IN)  and  p(xiSN).  (These  are  assumed 
to  be  measures  on  the  same  completely  additive  field  of  sets. )  The 
Radon- Nicodvrn  Theorem  states  that  there  is  a  function,  £(x),  and  a 
sot  of  N-measure  zero  Aq,  such  that  Eq.  1.  5  holds  for  any  mutually 
measurable  sot. 


To  or  the  N-measure  of  some  collection  of  observations,  C.  integrate 


r 
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the  number  one  with  respect  to  the  N-measure.  To  obtain  the  SN-measuro 
of  the  same  collections  of  observations,  integrate  the  weight  f(x)  with  res¬ 
pect  to  the  N-measure  and  add  on  the  SN-measure  of  C^Aq.  This  allows 
one  to  obtain  both  measures  by  integration  with  respect  to  p(x  IN), 
except  for  sets  of  N-measure  zero.  The  function  1 £(x)  is  never  really 
utilized  when  x  is  in  the  set  of  N-measure  zero,  Aq.  We  may 
define  f  (x)  to  be  im.nitv  on  this  set  as  a  notational  convenience.  Such  a 
function  C(x)  is  called  "the  Radom-Nicodym  derivative  of  the  SN-measure 
with  respect  to  the  N-measure.  "  and  is  the  most  general  form  of  a  like¬ 
lihood  ratio. 

The  vast  majuriiy  of  the  effort  in  the  physical  theory  of 
signal  detectabTity  is  devoted  to  obtaining  explicit  equations  for  0(x), 
and  to  realizing  practical  equipment  which  will  compute  its  value.  In 
contrast,  this  present  work  is  devoted  to  evaluating  performance  and  not 
to  obtaining  performance.  The  meaning  of  the  terminology  "a  decision 
is  based  on  likelihood  ratio  with  cut  value  (3  and  boundary  value  r"  is 


i 
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of  the  ROC  region  Compare  its  performance  to  the  performance  of 
any  other  possible  decision  function,  which  is  labeled  h(x).  The 
proot  is  shortest  if  we  consider  the  contrived  function  Q^. 

Q  =  [  P("A  ”  ISN)  -  P(”A”|SN)  ]  -  /3[  P("A  "  I N)  -  P("A  "  IN)  ] 
g  h  g  h 

(1.7) 

Q  is  a  comparison  of  the  probabilities  of  detection  for  the  g  and  h 

p 

decision  devices,  together  with  a  /3-weighted  comparison  of  the  proba¬ 
bilities  of  fal-e  alarm  of  the  two  devices.  The  probabilities  of  false 
alarm  are  given  by  direct  integration  with  respect  to  the  N-measure, 


P("A  "IN)  =  f  g(x,  i3,  r)  dp  (x  IN) 

g  '  *-Ao 

> 

P("A  "IN)  -  /  h(x)d/i  (x|N) 


(1.8) 


where  X-Aq  means  those  observations  that  are  not  in  the  set  of 
N-measure  zero.  Aq.  The  probabilities  of  detection  could  be 
obtained  by  similar  integration  with  respect  to  the  SN- measure. 
However,  utilizing  the  Radon-Nicodym  Theorem  and  likelihood  ratio 
we  may  evaluate  the  probabilities  of  detection  by  again  using  the 
N-measure. 


P("A  "iSN)  -  J  g(x.  ,3.  r)  C(x)d/i  (x  IN)  4  f  g(x,  ,3,r)d ti  (x  ISN) 

X-A0  Aq 

(1.  9) 

Pi"A  "iJN)  i  h(x)  ((x)dp  (x  IN)  4  /•  h(x)dp(x.SN) 

^  V  A  A 
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The  six  integrals  in  Eqs.  1.  8  and  1.  9  can  be  collected  into  two  integrals 
for  determining  the  value  of  the  quantity  Q^. 


Q  =  /  [g(x,,J,  r)  -  h(x)  ]  l*(x)  -  /3]d/i(xlN) 

X-  AQ 

+  f.  [g  (x,  3,  r)  -  h(x)  j  d/i  (x  ISN) 

•  A„ 


(1.10) 


The  first  integrand  is  a  product  of  two  factors.  When  the  likelihood 
ratio  is  greater  than  ^3,  the  second  factor  will  be  positive.  Since  g 
is  one  for  these  observations,  the  first  factor  will  be  zero  or  positive 
For  those  observations  w’th  likelihood  ratio  less  than  3.  the  second 
factor  will  be  negative.  Since  g  is  zero  for  these  observations,  the 
first  factor  will  be  zero  or  negative. 


<(x)  3,  [g  -  h]  U  -  3  ]  =  [  1  -  h]  [£  -  ,1  ]  ^  0 

C(x)  =  [  g  -  h]  [  i.  -  3  ]  =  [  r  -  h]  [  0  j  =0  (1.  11) 

f(x)  ■  ,3,  (  g  -  h]  [f  -  3)  =  [0  -  h]  [f  -  3  }  >  0 


Since  the  integrand  of  the  first  integral  is  never  negative,  the  integral 
will  also  be  nonnegative.  We  now  turn  our  attention  to  the  second 
integral,  over  the  set  of  N- measure  zero.  The  likelihood  ratio  is 
infinite  for  these  observations.  Therefore,  g  is  one  and  the 
integrand  is  nonnegative. 
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We  have  therefore  determined  that  a  quantity 
is  nonnegative  for  the  decision  device  based  on  likelihood  ratio  whose 
cut  level  is  the  same  number  ft.  How  does  this  establish  that  the 
decision  based  on  likelihood  ratio  is  better  ?  Pick  any  particular 
decision  device.  Compare  it  with  a  decision  device  based  on  likeli¬ 
hood  ratio  which  has  the  same  probability  of  false  alarm.  Proof 
that  we  can  find  this  decision  device  based  on  likelihood  ratio  with 
matching  false  alarm  probability  is  given  in  Appendix  A.  Granting 
that  we  can  find  such  a  matching  device,  it  will  have  some  value  of 

ft  and  r.  The  quantity  Q  evaluated  for  that  specific  value  of  ft 

I' 

will  be  nonnegative.  Since  the  false  alarm  probabilities  are  equal, 

the  quantity  Q  is  simply  the  difference  between  the  two  detection 

probabilities.  Qa  being  nonnegative  means  that  the  detection 
p 

probabilities  for  the  decision  device  based  on  likelihood  ratio  is 
greater  than  or  equal  to  the  detection  probability  for  the  decision 
device  h. 


P("A  MlN)  =  P("Ah"lN) 

(1.13) 

q  >0  --£>  P("A  "  ISN)  >  P("A.  "  (SN) 
ft  —  g  —  n 

This  establishes  that  the  performance  of  .  .ion  based  on  likeli¬ 

hood  ratio  falls  on  the  optimum  ROC  cu*  .e  io*  ROC  region. 

The  ’’onto”  part  of  the  proof  establishes  that  for  every 
possible  probability  of  false  alarm  there  is  at  least  one  decision 


device  based  on  likelihood  raiio  which  yields  this  probability  of  false 
alarm.  This  part  of  the  proof  has  been  omitted  here  and  given  in 
Appendix  A  because  it  is  tedious  and  not  illuminating.  The  first  part  of 
the  proof  has  been  displayed  to  ouphasize  that  the  optimum  ROC  curve, 
the  upper  boundary,  is  directly  r rated  to  likelihood  ratio. 

i.  2  Fundamental  Theorem 

1.  2.  1  Likelihood  R?  io.  A  decision  axis  can  be  con¬ 
sidered  as  an  observable.  Whatever  the  physical  nature  of  the  original 
observation  and  the  relationship  of  the  decision  axis  to  these  original 
observations,  there  will  be  certain  probabilities  of  occurrence  of  the 
various  decision  values  under  each  of  the  two  classes  of  causes.  The 
question  immediately  arises,  "If  we  have  two  measures  induced  on 
this  decision  axis,  what  is  the  Radon- Nicodym  relationship  between 
these  two  measures?"  That  is,  "What  is  the  likelihood  ratio  of  the  deci¬ 
sion  value?" 

We  will  consider  the  special  decision  axis  which  is  the 
likelihood  ratio  of  the  observation,  and  will  establish  that  the  likeli¬ 
hood  ratio  of  this  decision  axis  is  numerically  the  same  as  the  value 
of  the  decision  axis.  That  is,  the  likelihood  ratio  of  the  likelihood 
ratio  is  the  likelihood  ratio.  For  example,  assume  an  observation  space 
and  a  device  which  numerically  evaluates  the  likelihood  ratio  of  each 
observation.  Let  us  further  assume  that  for  a  given  observation  the 
value  calculated  by  this  device  is  the  number  2. 74.  We  will  establish 
in  this  section  that  in  this  case  the  likelihood  ratio  of  the  number  2. 74 
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is  2.74.  That  is.  the  numerical  value  2.74  occurs  under  the  cause 
SN  with  probability  2.  74  times  the  probability  it  will  occur  under 
the  cause  N. 

Let  us  consider  an  observation  space,  X,  and  a  col¬ 
lection  of  measurable  sets  A  which  form  a  a-ring*J?/on  X.  A 
probability  function  is  any  nonnegative  completely  additive  set  function 
on  such  that  the  probability  of  the  entire  observation  space  is  unity. 
To  introduce  likelihood  ratio  we  consider  two  such  probability  functions 
defined  on  the  same  a- ring  ^ ,  and  compare  probabilities.  Repeating 
the  Radon-Nikodym  Theorem  (Ref. 2), 

If  p(AiSN)  is  a  completely  additive  set  function  on  a 
•7-ring  of  sets  which  are  measurable  with  respect  to  p(A!N),  then  there 
exists  a  unique  decomposition  consisting  of  the  following: 

Aq,  a  set  of  measure  zero:  p(AglN)  =  0 

f,  a  function  integrable  with  respect  to  p(xlN),  unique  almost 
everywhere 

Q,  a  set  function  defined  on  Aq 
such  that 

P(AISN)  =  /A  f(x)dMxlN) +Q(A°A0)  (1.14) 

Here  the  Radon-Nikodym  derivative  relates  p(-  iSN)  to  p(-  !N) 
except  for  a  set  of  N-measure  zero. 

Using  the  above  formulation  define  a  likelihood  ratio 


f(xl  as 
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b 


£(x)  =  f(x)  x  f.  A 


0 


(1.15) 


-  X) 


x  r  A, 


The  definition  of  likelihood  ratio  is  unique,  up  to  a  set 
of  combined  measure  zero.  That  is,  many  functions  and  sets  AQ 
may  satisfy  the  definition;  however,  for  any  two  likelihood  ratios  let 


be  any  measurable  set  on  which  they  differ.  Then 


,'Ad|SN)  +  m(Ad|N)  =  0 


(1.  16) 


Corollary 


P(AlSN)  -  j  £(x)d/i  (x  I  N)  +Q(A°A 


A-A 


0 


P(A  I N)  =  /  dp(xlN) 

A“A0 


}  (1.17) 


Let  us  consider  the  implications  of  this  definition  to  the  familiar 

situations  of  probability  density  functions  and  point-mass  probabilities. 

For  example,  consider  an  n-dimensional  space.  Let 

x  -  (x,.  x  ,  .  .  x  )  and  let  there  be  density  functions.  Then  for 
i  z  n 

anv  Borei  sei  A 


*  \ 


P(AlSN)  =  j.  f(x!SN)dx 


P(AlN)  JAf(x:N)dx  (1.18) 


Let  Aq  be  those  points  for  which  the  N  density  vanishes.  Then  for 
any  Borei  set 


1  f 
5 

1  i 


— -  * 
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P(AiSN)  =  P(A- AQ  I  SN)  +  P(A0nAlSN)  (1.19) 


P(AlSN)  =  f  f(x|SN)dx  +  P(A  nAlSN) 
A-A0 


(1.  20) 


Multiplying  and  dividing  by  f(xlN) 


P(A.SN)  =  j  f(x:N)dx+ P(A0nAlSN) 

A- A„ 


while 


P(AlN)  -  /  f(xiN)dx 

A-Art 


Matching  terms  w'.th  Eq.  1.  17 


(1.2?) 


£(x)  =  flx  '  iff(x|N)  t^O 


f(x  IN) 


if  f(xlN)  =  0 


(1.  22) 


Next  consider  a  point  space  Jx.l,  and  A  be  any  subset 


of  these  points.  Then 


P(A;SN)  2,  P(x  ISN)  ,  P(AlN)  =  2,  P(x.  |N)  (1.23) 

x.t  A  1  x  £  A  1 

i  l 


Let  Aq  be  those  points  having  N- probability  zero, 


P(AISN)  ^  P(x.  ISN)  4  P(A°AJSN) 
*  x.  >  A-An  1 

j  0 


(1.24) 
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P(A.'SN) 


v 

LJ 

x.  e  A-A 
1 


P(x.lSN) 
P(x  I  N) 


P(x.lN)  +  P(AnAQISN) 


while 


P(Ai'N)  V  p(x..N) 

VA-A0  1 

Matching  terms  with  Eq.  I.  17 

if  P(x.lN)  £  0 
if  P(x.lN)  =  0 


P(x.  |SN) 
((xi'  "  ~P(x.  1 N) 

-  oo 


J 


(1.26) 


We  shall  not  continue  with  the  special  cases  by  treating  spaces  with 
distribution  functions,  leading  to  a  mixture  of  densities  and  point-mass 
probabilities.  The  purely  mechanical  difficulties  involved  in  notation 
make  the  general  Radon- Nikodym  formulation  attractive. 

Before  proving  the  basic  theorem  that  "the  likelihood  ratio  of  the 
likelihood  ratio  is  the  likelihood  ratio .  ”  let  us  set  up  the  proper  frame¬ 
work  of  "induced  measures. " 

1.  2,  2  Induced  Measures.  Paraphrasing  from  Pitt,  (Ref.  2,, 

PP- 25  -27)  let  a  be  any  function  mapping  X  onto  Y.  not  necessarily 
one-to-one,  iVa  cr-ring  of  sets  in  X  and  n  a  measure  on  it.  We  will 
say  a  set  B  in  Y  is  a  simple  set  if  a  *(B)  is  in  the  o-ring  «*/.  Obviously, 
the  collection  of  all  such  sets  B  form  a  o-ring  &  The  function  v  is 
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called  "the  measure  induced  in  Y  by  p  and  o"  il 

Bt&,  KB)  =  h{«\b)}  (1.27) 

Integration  of  a  function  in  Y  can  be  reflected  back  to  the  original  space 
X.  Quoting  Pitt,  Theorem  3 1 : 

Theorem  31:  Suppose  p  is  a  measure  in  X  and  v  is  the  measure  induced 
in  Y  by  y  -  g(x).  Then  for  functions  of  y 

/f(y)dv  =  [f(a(x))d  p  (1.28) 

Y  X 

in  the  sense  that  if  one  integral  exists,  then  so  does  the  other  and  the  two 
are  equal. 

To  say  that  "B  is  a  : -measurable  set"  is  equivalent  to  saying 
that  the  characteristic  function  lQ(y)  is  integrable  with  respect  to  v. 

D 

iB(y)  -  1  yf  b 

=  0  yf  B 

r-(B)  l  1  (y)d^(y)  =  |  du  (1.30) 

Y  B  B 

Since  the  product  of  integrable  functions  is  also  integrable,  for  any  integrable 

f  of  Theorem  31.  1^,  times  f  is  also  integrable.  We  therefore  have  the 

13 

corollary: 

Corollary  to  Pitt’s  Theorem  31:  Suppose  p  is  a  measure  in  X  and  v 


is  the  measure  induced  in  Y  by  y  =  u(x);  then  for  any  measurable  set 

B*  3B 
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I  f(y)di/  -  /  f(Qf(x)  )d/i 


a  1(B) 


if  cither  ff(y)di/  or  f  f(  «(x)  )  dju  exist. 
Y  X 


(1.31) 


Let  us  diagram  our  current  status.  We  have  assumed  an 
observation  space  X,  a  a-ring  of  measurable  sets,  two  measures 
p(xlN)  and  p(xlSN).  For  any  map  we  can  consider  the  induced  measures 
v(  r  IN)  and  v( r  |SN) .  If  the  map  is  the  likelihood  ratio  £(x),  the  image 
of  X  is  some  portion  of  +Le  reals,  together  with  oo. 


f;  X 


R  c  E’U  |oo| 


B  measurable 


measurable 


(:  ji (x  IN)  —  i'(rlN) 

(:  p(x  ISN)  — ►  i/(rlSN) 


Right  away  let  us  single  out  r  =  oo  and  the  special  set 


and  list  their  properties. 


:  Aq  ►  *>  ,  «'  (  {«>[  )  =  Aq 
p( Aft  IN)  =  0  i/(»  IN)  =  0 

U 


p(AqISN)  exists  ~>  ^(aorSN)  is  some  number. 


For  anv  measurable  B  of  reals 


2  b 


C_1(B)  °  Aq  =  0  if  »  %  B 

r1(B)nA.  -  An  if  oo  c  B 
U  0 

We  now  have  a  space  R  with  a-ring  of  measurable  sets, 
and  two  measures  u(r  IN)  and  i/(rISN).  The  question  arises,  What 
is  the  Radon- Nikodym  derivative  of  v  (r  |SN)  with  respect  to  i^r  |N)? 
What  is  the  likelihood  ratio  on  R  space?  To  answer  this  we  consider  the 
identify  map  i(r)  which  maps  each  point  into  itself. 


X - - - -  R— > — -  R 

( 

By  definition  the  space  X  has  probability  one  under  both  v  measures. 


1  =  m(X[SN)  I  £(x)dp(x  IN)  +  p(A  ISN)  (1.32) 

x-'Ao 


Because  (  is  integrable  with  respect  to  p,  for  any  set  B  measurable 
in  R  space 


J 

(B)-  Ao 


t(x)dp(x 


IN) 


(1.33) 


exists. 

If  we  now  consider  the  identity  map  of  R  onto  itself,  for  any 
measurable  B 


J  i(r)d  v(r  I N)  =  /  i(£(x)  )dp(  x  IN) 


(1.34) 
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t 


Because  i(£)  -  £ 


J  i(r)d  u{r\  N) 
B-  x 


«"Wa, 


£(x)d/i(x  IN),  which  exists  (1.35) 


=  m(£_1(B)  -A0  ISN)  bydef.  of  £  (1.36) 

=  r  (B- jooj  I SN)  bydef.  of  v  being  an  (1.37) 
induced  measure 


Therefor  e  we  may  write 


r^BiSN)  -  /  i(r)d^(r|N)  +  r^(Bn  x  ISN) 

B- jx| 


(1.38) 


By  direct  identification  i(r)  is  a  likelihood  ratio.  We  have  therefore 
shown  the  following  theorem: 

Theorem:  A  likelihood  ratio  of  r  (which  is  a  likelihood  ratio  of  x)  is 
r  itself. 


We  complete  this  formal  abstract  work  with  one  further  con¬ 
sideration.  Consider  any  one-to-one  transformation  of  R  which  maps 
infinity  onto  infinity.  This  means  take  any  function 


w  =  u(r) 


(1.39) 


and  its  inverse 


r  =  v(w) 


(1.40) 


such  that 


1  j 

i 
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u(ooj  =  x>,  v(oo)  =  00  (1.41) 

Of  course 

W  -  u^R )  (1.42) 

Let  the  induced  measures  on  W  be  A(w|N)  and  A(w|SN).  The  A- mea¬ 


surable  sets  ^  are  the  image  of  m 

-  U<#)  (1.43) 

For  any  measurable  C  e  ^ ,  let  B  =  v(C).  Then 

A  (C  i  SN)  =  \(C  -  {ooflSN)  +  A(C°{oo|  1SN)  (1.44) 

A(CSSN)  =  i/(B  -  |oo|  ISN)  +A(C°|  »}  ISN)  (1.45) 

A(CiSN)  -  f  rdy(r  IN)  +  A(C°{oo}  lSN)  (1.46) 


b-M 

Bv  the  corollary  to  Pitt's  Theorem  31 

/  rdy(r|N)  =  J  v(w)dA(wiN)  (1.47) 

B-{oo}  C-{oof 

So 

A(CISN)  =  f  v(w)dA  (wlN)  +  A  (C^xf  iSN)  (1.48) 

c-i=°l 

We  see  immediately  that  v(w)  is  a  likelihood  ratio. 

Example  If  w  In  r.  then  r  -  eW.  If  r  is  a  likelihood  ratio, then 

w 

e  is  a  liKPlihood  ratio  for  w  In  r. 
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1.  2.  3  Section  Sun  unary.  wnai  has  been  considered  is 
diagrammed  in  Fig.  1.  6.  from  any  observation  space  X  with  mea¬ 
sures  ji  and  special  set  there  is  a  likelihood  ratio  £(x)  such 
that 

g(AlSN)  -  f  £(x)d/i(xlN)  + /i(AnA  iSN)  (1.49) 

A-A0 

The  induced  measures  on  R  =  £(X)  are  v ,  so  related  that  their 
likelihood  ratio  is  simply  the  value  of  r  itself. 

WBISN)  =  /  rd^r  IN)  4  i.(Bnja)llSN)  (1.50) 

B-  1=0} 

For  any  one-to-one  transformation  which  leaves  oo  fixed,  say 
r  =  v(w) 

A(ClSN)  -  I  v(w)dA  (wlN)  +  A  (Cn  joo  }  |3N)  (1.51) 

c-{*} 

The  likelihood  ratio  of  w  is  simply  the  "substitution  value"  v(w). 

In  order  to  establish  the  proofs,  care  has  been  taken  to  avoid  notation 
which  is  "suggestive. "  With  the  proof;;  complete  we  may  now  make 
use  of  more  suggestive  notation  to  summarize  the  two  conclusions. 

£(£)  =  f  (1.52) 

£(w  (£)  )  -=  w  ^  (w )  (1.  53) 

and  in  particular  if  z  =  In  £ 
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r 


I 


i 


- -  r  =  £(x)  — •>  -  w  =  w(r)  — •> 

* —  r  =  v(w  ')  - 


x  R  w 


/i(xlN),  /x(xlSN),  f(x)  j/(r I N),  */(r|SN),  i(r)  X(w|N).  X(wlSN),  v(w) 


i 

i 


Fig.  1.  6.  Sketch  of  spaces  and  transformations 


r 
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f(z)  =  eZ  (1.54) 

When  the  measures  y(£|N),  v(£|SN)  are  given  by  distribution 
functions,  we  may  write 

J 

F(£|SN)  -  /  £dF(£|N)  (1.55) 

0 

This  special  case  was  given  in  Ref.  1  as  Theorem  8,  and  was 
written 


dF(/3|SN)  =  ,3dF(/3lN) 


(1.56) 


1.  3  Initiation  of  Present  Study 

In  1954  W.  W.  Peterson  (Ref.  1,  pp.  205-206)  showed 
the  rather  amazing  theorem  that  if  the  logarithm  of  the  likelihood  ratio 
is  normally  distributed  under  condition  N,  then  it  is  also  normally 
distributed  under  SN.  Specifically,  he  started  with  the  assumption 
z  --  In  f  has  a  normal  probability  density  function  under  N. 

(z-m)2 

f<ziN)  =  TOT"  e  2d  <L57) 


/ 

i 


He  was  first  able  to  show  that  the  mean  is  related  to  the  variance, 
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and  finally  that  the  SN  probability  density  of  the  logarithm  of  the 
likelihood  ratio  is 

.  (z  ~  -5d)" 

f|z;sN)  ‘  tuft  e  2d  a.  59) 

Equations  1.  58  and  1.  59  may  be  expanded  to  show  that  the  likelihood 

z 

ratio  of  2  is  indeed  e  . 

,  f(z  SN)  z 

-  f(z  N)  ~  e  (1.60) 

This  theorem  of  Peterson  established  the  basis  for 
the  one  parameter  family  of  ROC  curves,  the  normal  POC  curves.  The 
curves  are  indexed  by  the  single  real  parameter,  d,  or  as  it  is  done 
m  the  psychophysical  literature  by  the  single  parameter  d'  = 

This  one  parameter  family  of  ROC  curves  has  been  used  extensively 
in  both  the  electrical  engineering  and  psychophysical  work  of  detection. 

The  doubly  truncated  Halsted  distribution  is  a  five 
parameter  class  of  probability  density  functions.  The  untruncated  dis¬ 
tribution  lias  been  used  in  the  study  of  rapidly  fading  signals  (Ref.  3). 
It  arose  again  in  a  study  of  signal  detection  and  learning  in  which  the 
amplitude  of  the  signal  was  initially  unknown  (Ref.  4).  When  it  is 
used  as  the  N-probability  density  function  for  the  logarithm  of  the 
likelihood  ratio, it  is 

_  z2 

f(2  IN)  =  F0zBe"CZ  e  2D  A-.Z<E  (1.  bl) 


It  was  discovered  that  the  SN- probability  density  function  for  the 
logarithm  of  the  likelihood  ratio  fell  in  the  same  class 

_  z' 

f(zlSN)  -  F1zBe'(C“1)Z  e  2D  A  <  z  <  E  (1.62) 

Since  this  class  of  functions  contains  as  subclasses  the  exponential, 

the  normal,  the  Rayleigh,  the  chi-square,  and  the  chi  distributions 

as  special  cases,  it  was  evident  that  many  theorems  like  Peterson's 

normal  theorem  could  be  obtained  from  this  relation.  The  key  to  all 

these  theorems  is  Eq.  1.  54  ,  that  the  likelihood  ratio  of  the  logarithm 

z 

of  the  likelihood  ratio  is  e  . 
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ROC  CHARACTER 


2.  1  ROC  Curve 

In  tne  introduction  a  simple  decision  process  was  charac¬ 
terized  as  one  which  assigns  a  probability  p("A"  i  x  )  to  each  obser¬ 
vation  x,  denoting  the  probability  that  this  decision  mechanism  will 
give  the  response  "A"  wnenever  x  is  observed.  The  performance 
of  sucn  a  decision  device  in  a  specific  observation  situation  plots  as 
a  signal  point  on  the  ROC  plot.  The  mechanism  for  decisions  based 
upon  likelihood  ratio  considered  a  probability  function  g(x,  f3,  r) 
which  yielded  an  entire  ROC  curve.  The  parameters,  /i  and  r  , 
added  sufficient  dimensionality  to  the  simple  decision  function  to 
produce  this  complete  curve.  In  general,  decision  functions  are 
parameterized  by  some  index  wnich  allows  more  than  one  performance 
point.  If  the  dimensionality  of  the  index  set  is  too  small,  tne  decision 
mechanism  may  generate  only  a  set  of  points,  such  as  shown  in  Fig.  2.  1(a), 
or  it  may  generate  a  continuum  of  points  leaving  some  gaps  in  the  curves, 
or  possibly  multiple-values  for  y  as  a  function  of  x,  as  shown  in 
Fig.  2.1(c). 

These  differences  between  the  types  of  ROC  curve  given 
by  decisions  based  upon  likelihood  ratio  and  decisions  which  give  incom¬ 
plete  or  multiple-valued  plots  on  the  ROC  curve  may  be  removed  by  a 
process  of  convex  completion.  The  graphic  form  of  convex  completion 


ISN)  h-  o  P(  "A" 1 SN) 


x  -  PC'A'' I N)  x=P("A”lN) 


(c)  Original  arcs  (d)  Convex  completion  of  (c) 

Fig.  2.  1.  Illustration  of  complex  completion  of  ROC 


0  x  =  P("AMlN)  1 

r 

Fig.  2.2.  Illustration  of  tangent  cone  at  one  point 
of  an  ROC  curve 


was  to  add  to  the  ROC  region  the  l.ne  segment  joining  any  two  possible 
points  available  from  agiven  decision  mechanism.  By  such  a  process, 
a  single  point  not  on  the  char.ce  diagonal  yields  a  whole  parallelogram  of 
points  surrounding  the  chance  diagonal. 

Figure  2. 1  illustrates  two  cases  of  convex  completion. 

The  first,  illustrated  in  Figs.  2. 1  (a)  and  2. 1  (b),  show  the  results  of 
convex  completion  for  a  finite  set  of  points.  In  Fig.  2  1  (b),  only  the 
upper  bound  of  the  ROC  region  has  been  filled  in.  This  is  sufficient, 
since  the  lower  bound  of  the  ROC  region  lies  below  the  chance  diagonal 
and  could  be  obtained  by  symmetry  through  the  midpoint  if  desired.  This 
uppei  b'.ona  lies  on  or  above  all  of  the  internal  secants  forming  the  com- 
pi f  \  npletion  between  any  two  possible  points.  In  Fig.  2.  1  (c)  a 
,  n«  v  f.  t  different  case  is  involved.  Between  the  points  with  false  alarm 
p  .^abilities  x^  and  Xg,  there  is  a  gap  which  is  filled  in  with  the  convex 
completion  of  those  two  points.  Since  the  result  is  better  (that  is,  above 
and  to  the  left)  than  the  short  segment  of  arc  in  the  interior  of  the  region, 
this  short  segment  disappears  from  consideration.  Secondly,  the  left- 
hand  side  of  the  original  arc  shows  a  doubling  back  of  the  arc,  making 
the  function  y(x)  triple  valued  on  a  region  of  false  alarms.  A  straight 
line  can  be  drawn  from  the  point  (0,  0)  tangent  to  the  original  curve  in 
the  triple  valued  section,  that  exceeds  all  of  the  original  curve  up  to  the 
point  of  tangency.  The  resultant  region  shown  in  Fig.  2.  1  (d)  exhibits  a 
smooth  upper  bound,  the  ROC  curve. 

The  ROC  curve  lies  on  or  above  any  internal  secants 
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since  it  is  the  upper  bound  of  ail  convex  completions.  A  convex  ROC 
curve  also  lies  on  or  below  any  tangents  or  tangent  cones  drawn  to 
points  on  the  ROC  curve.  For  a  tangent  to  one  point  on  the  ROC  curve 
to  contact  another  point  of  the  ROC  curve  would  require  that  the  convex 
completion  of  those  two  points  would  fall  above  the  ROC  curve.  Since 
the  ROC  curve  is  on  or  above  all  internal  secants,  this  situation  can¬ 
not  occur.  Figure  2.  2  illustrates  a  tangent  cone  drawn  at  one  point 
of  an  ROC  curve.  At  that  point  the  curve  is  continuous  but  has  a  break 
«  in  slope,  so  that  there  is  no  unique  tangent  line  to  that  point.  The  tan¬ 

gent  cone  consists  of  all  of  those  lines  contacting  the  ROC  curve  at  only 
that  one  point,  otherwise  lying  everywhere  above  it. 

The  upper  bound  of  the  convex  completion  of  given  ROC 
points  is  the  ROC  curve  for  those  points  or  segments.  Such  an  ROC 
curve  is  characterized  by  being  complete  and  convex.  The  mathemati¬ 
cal  adjective  "complete"  means  that  for  each  value  of  the  horizontal 
axis  there  is  one  value  for  the  vertical  axis.  That  is,  the  function 
y(x)  is  a  well-defined  single  value  function  for  x  between  zero  and  one. 
Definition 


Complete:  y(x)  has  a  single  value  for  each 

x  (  L  0,  1  ] 


(2.1) 


Convex  means  that  the  values  of  the  curve  lie  on  or  above  the  internal 


s<  cants. 
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Definition 

Convex:  •_  ^  x^  ,  0  <_  a  _  1 

(2.2) 

y(x2)  >  nytXj)  +  (1  -  n)  y(x3) 

The  term  "singular"  has  been  used  in  the  detection 
literature  to  mean  perfect  performance.  In  terms  of  the  ROC  curve 
this  means  that  the  probability  of  detection  is  unity  for  all  probabilities 
of  false  alarm.  This  is  illustrated  in  Fig.  2.3(a). 


XX  X 

(a)  Singular  (b)  Nonsingular  (c)  Regular 

Fig.  2.  3.  Illustrating  singularity  for  complete  convex  ROC  curves 


Definition 

Singular:  y(x)  =1  for  all  xt  [0,  1]  (2.3) 

Once  it  has  been  established  that  a  given  situation  leadt, 
to  singular  detection  there  is  no  need  for  further  description  of  its  per¬ 
formance.  A  curve  may  be  nonsingular  and  still  reflect  a  certain 
degree  of  perfect  detection.  That  is,  there  may  be  some  observations 
yielding  a  positive  probability  of  detection  and  a  zero  probability  of  false 
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alarm.  Correspondingly  there  may  be  a  set  of  observations  for  which 
one  can  unequivocably  respond  "B,  "  for  which  the  probability  of  a 
miss  is  zero.  The  ROC  for  the  hypothetical  situation  containing  both 
of  these  possibilities  is  shown  in  Fig.  2.  3(b).  Such  a  curve  will  be 
considered  complete,  convex,  nonsingular,  and  nonregular.  A  regular 
ROC  curve  is  complete,  convex,  and  interior  to  the  unit  square  ex¬ 
cept  at  ihe  chance  points  (0,  0)  and  1,  1). 

At  this  point  the  precise  statement  of  the  type  of  ROC 
curves  that  will  be  considered  can  be  made.  This  work  will  deal 
only  with  complete,  convex,  nonsingular  ROC  curves,  or  with  non¬ 
singular  ROC  curves  that  may  be  made  complete  and  convex  by  the 
process  of  convex  completion. 

Definition1 

Regular:  complete,  convex,  x  =  0  =>  y  =  0 

(2.4) 

y  =  i  =>  x  =  1 


2.  2  ROC  Models 


In  this  section  two  propositions  are  proved.  The  first 
proposition  is  that  an  ROC  curve  contains  insufficient  information  to 
specify  the  observation  statistics  that  ied  to  it,  or  even  to  specify  the 
statistics  on  a  real  decision  axis  that  lead  to  it.  The  second  proposition 


is  that  an  ROC  curve  does  contain  sufficient  information  to  specify  the 


statistics  on  the  logarithm  of  the  likelihood  ratio. 

2.  2. 1  One  ROC  Curve.  Many  Models.  The  purpose  of 


this  section  is  to  demonstrate  that  a  single  ROC  curve  contains  insuffi- 


3>  is  read  "implies 


cient  information  to  specify  the  observation  statistics  or  decision  axis 
from  which  it  was  obtained.  Rather,  there  are  many  decision  axes  and 
observation  statistics  leading  to  the  same  ROC  curve. 

To  begin  the  demonstration,  start  with  a  real  variable, 

£  ,  and  choose  a  distribution  function,  F^(£  ),  fairly  arbitrarily.  We 
shall  also  assume  that  a  .regular  ROC  curve,  Y(X),  has  been  selected. 

The  random  variable  £  will  be  used  as  a  decision  axis,  and  F^(£  )  will 
be  used  as  the  N  distribution  function.  The  SN  distribution  function  for 
£  will  be  chosen  to  obtain  an  ROC  curve  identical  to  the  given  ROC  curve. 


’B"  •*- 


’0 


V 


£ 


Fig.  2  4.  Arbitrary  decision  axis,  £ 


To  say  that  the  £  -  axis  is  a  decision  axis  means  that 
whenever  a  particular  value  of  £  occurs  which  fails  above  a  cut  level, 
say  £  .  the  decision  will  be  "A.  "  In  contrast,  when  it  falls  below  £  \ 
the  decision  will  be  "B„  "  When  the  observation  falls  right  at  the  value 
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£  '  a  second  (randomizing)  random  variable  is  introduced  to  determine 
how  often  the  decision  "A"  is  elicited.  Since  the  purpose  of  this  develop¬ 
ment  is  to  show  that  there  are  many  possible  ways  to  set  up  observation 
statistics  on  £  ,  let  us  introduce  some  convenient  restrictions  on  these 
observation  statistics.  To  avoid  limiting  considerations,  it  is  assumed 
that  the  probable  values  of  £  fall  in  the  open  interval  between  two  num¬ 
bers  £  q  and  £  j.  Therefore,  the  distribution  function  is  zero  below 
£  Q  and  is  equal  to  one  at  and  above  £  ^.  It  is  also  convenient  to  assume 
that  the  distribution  function  is  strictly  monotone  increasing  in  the  inter¬ 
val  between  £  and  £  This  guarantees  that  if  a  particular  value  of 
Fj  between  zero  and  one  is  assumed  for  some  £  value  that  we  can 
uniquely  solve  for  this  £  value. 


Fj(0  -  0 


£  <  £ 


Fj(£  )  is  strictly  monotone  increasing  £  ^  <  £  <  £  j 


(2.  5) 


Fj(£  )  =  1 


With  these  minor  restrictions  established,  let  us  set  the 


N  distribution  equal  to  F 


F(£  IN)  =  FjU) 
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It  will  be  convenient  to  have  a  special  symbol  for  one  minus  the  distri¬ 
bution. 


XU)  =  1  -  FjU) 


(2.  7) 


X  at  the  largest  value  of  £  ,  £  ^ ,  is  zero.  X  at  the  smallest  value  of 
£  ,  £  is  one.  X  is  strictly  monotone  decreasing  as  a  function  of  £ 
on  the  interval  X  between  zero  and  one. 


XU.)  =  0  x($  0)  =  i 

X  sJ/£  on  0  <  x<  1 


(2.  8) 


(The  notation  X  sL  £  means  "strictly  monotone  decreasing.  ") 

The  decision  axis  and  the  distribution  on  £  under  N  were  arbitrary. 
We  now  choose  the  observation  statistics  for  £  under  the  condition 
SN,  using  the  specified  regular  ROC  curve  Y(X). 


FU  ISN)  =  1  -  Y[XU)J  i  < 


=  1 


4  >  « 


(2.9) 


The  N  Distribution  function  is  certainly  a  legitimate  distribution.  Is 
the  SN  function  as  given  a  legitimate  distribution  function?  For  values 
of  the  variable  less  than  the  minimum,  £  the  X  value  is  one  and  hence 
the  Y  value  is  one,  since  Y(l)  =  1.  Therefore,  the  distribution  function, 
under  the  condition  SN,  is  zero  for  all  small  values  of  the  argument  £  . 
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£  <  £  Q.  X(£)  =  1.  Y  =  1.  F(£  iSN)  -  0  (2  10) 

For  intermediate  values  of  the  argument  £  ,  X  is  a  strictly  monotone 
decreasing  function  of  £  .  Y  is  a  monotone  increasing  function  (not 
necessarily  strictly  monotone  increasing)  function  of  X.  The  distri¬ 
bution  function  under  the  condition  SN  is  a  strictly  monotone  function 
of  Y.  Putting  these  all  together  we  see  that  the  distribution  function  for 
£  under  the  condition  SN  is  a  monotone  function  of  its  argument,  £  . 

£  Q  <  £  <  £  r  X  i  £  ,  Y]X,  F(£  !SN)  i  y,  F(£  ISN)  J  £  (2.  11) 

Now  we  have  a  decision  axis,  £  .  and  two  distribution 
functions  nr.  the  decision  axis  for  observation  statistics.  Let  us  deter¬ 
mine  the  ROC  curve.  We  will  use  x^  to  denote  the  values  of  the  proba¬ 
bility  of  false  alarm  plotted  along  the  horizontal  axis,  and  y2  to  denote 
the  values  of  the  probability  of  detection  plotted  along  the  vertical  axis. 
The  subscript  two  is  used  to  indicate  that  this  is  the  second  ROC  curve; 
the  first  one  is  the  original  or  given  ROC  curve  Y(X). 

The  probability  of  false  alarm  is  simply  the  N- probability 
that  the  decision  variable  will  exceed  the  threshold  value  £'.  which  is 
simply  one  minus  the  distribution  function  for  £'  under  the  condition  N. 

x2(£’)  =  Pr(£  >  £ '  1 N)  =  1  -  F(£'|N)  (2.12) 


The  distribution  function  of  £ '  under  N  condition  may  be  written  in 
terms  of  X(£  )  by  combining  Eqs.  2.  6  and  2.  7.  When  these  are 


substituted  into  Eq.  2. 12  we  obtain 


x2(o  =  i  -  [i  -  x(ni 

(2.13) 

x2(n  = 

The  probability  of  detection  is  derived  in  a  completely  analogous  manner. 

y2(n  =  Pr(£  >  HSN)  (2.14) 

y2(0  -  1  -  f  1  -  Y(X($'))i  =  YfX(r)|  (2.15) 


Equation  2.  13  can  be  used  to  justify  replacing  X  by  x2  to  obtain 

y2(H  -  Y[x2(Ol  (2.16) 

The  relationship  between  the  vertical  and  horizontal  axes  in  this  second 
ROC  is  exactly  the  same  as  it  was  in  the  original  ROC  curve;  that  is 
the  two  ROC  curves  are  identical. 

y2(x2)  s  Y(X)  (2.17) 

Let  us  review  the  above  argument.  Start  with  any  com¬ 
plete  convex  ROC  curve  for  which  a  model  is  desired.  A  model  is  a 
decision  mechanism  and  observation  statistics  which  yield  tnat  given  ROC 
curve.  The  decision  axis  must  meet  only  tne  minor  restriction  of  bounded¬ 
ness.  The  distribution  function  for  the  variable  under  the  condition  N 
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must  meet  only  the  minor  restriction  of  monotoneity.  (Both  restrictions 
were  for  convenience  and  are  not  strictly  necessary. )  Once  a  specific 
choice  of  these  two  has  been  made,  a  distribution  function  under  the  other 
condition,  SN,  can  be  specified  to  obtain  the  given  ROC  curve.  This 
result  has  two  implications.  First,  a  regular  ROC  curve  and  a  fixed 
decision  axis  together  contain  insufficient  information  to  specify  the 
observation  statistics  that  led  to  the  given  ROC  curve.  Second,  given  a 
regular  ROC  curve  for  which  a  model  is  desired,  one  may  exercise  great 
freedom  in  assuming  the  nature  of  the  decision  axis  and  one  distribution 

on  it. 

This  above  demonstration  required  that  the  ROC  curve 
be  regular.  This  could  have  been  relaxed  from  "regular"  to  "complete, 
convex,  nonsingular.  "  The  additional  complexity  did  not  seem  warranted. 
Ir,  Section  3.  5  a  functional  relation  will  be  established  between  a  complete, 
convex,  nonsingular  ROC  curve,  and  an  associated  regular  ROC  curve. 
After  that  section,  one  may  conclude  that  each  model  for  the  associated 
regular  ROC  curve  implies  a  model  for  the  complete  convex,  nonsingular 
ROC  curve,  thereby  obtaining  many  models  for  the  nonregular  ROC  curve. 
A  Notation  Introduced.  Many  of  the  equations  in  this  work  will  occur  in 
pairs  of  equations,  one  for  the  condition  N  and  the  other  for  the  con¬ 
dition  SN.  It  will  be  convenient  to  have  a  notation  that  signifies  both 
equations  when  they  are  nearly  alike.  The  notation  used  is  a  slight 
modification  of  the  ±  type  of  notation  commonly  used.  Its  use  is  shown 

in  the  following  two  equations. 

Consider  two  exponential  density  functions. 
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f(tiSN)  =  e 
f(t  IN)  =  e' 


it  I  <  .  48125 
It !  <  .  48125 


(2.18) 


In  the  double  notation  these  would  be  written 


f(t  S^)  =  e*1  It  I  <  .  48125 


(2.19) 


When  the  upper  condition  is  used,  the  upper  sign  is  used. 

Consider  the  following  equation  in  double  notation. 


Prob  (z  <  j3|??)  =  /  f(z  S3  dz 


(2.  20) 


The  upper  condition  is  used  throughout  the  whole  equation,  or  the  lower 
condition  is  used  throughout  the  whole  equation. 

2.  2,  2  A  Unique  Model  for  Each  ROC  Curve.  In  the  pr  e- 
vious  section  it  was  shown  that  there  are  many  models  for  each  regular 
ROC  curve.  In  this  present  section  the  relation  between  the  decision 
axis  and  the  likelihood  ratio  will  be  specified.  Under  this  condition, 
the  specification  of  the  ROC  curve  will  uniquely  determine  the  distri¬ 
butions  on  the  decision  axis.  The  demonstration  is  complicated  enough 
for  regular  ROC  curves;  the  inclusion  of  consideration  of  the  nonregular 
but  complete,  convex,  nonsingular  ROC  curves  adds  no  further  complexity. 

The  specif  ic  decision  axis  considered  is  the  z-axis,  where 
z  is  the  logarithm  of  the  likelihood  ratio  of  the  observation. 


z  =  In  £(X) 


(2.21) 
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We  shall  first  analyze  an  ROC  curve  derived  from  distributions  on  the  z- 
axis,  then  discuss  the  synthesis  of  z-axis  distributions  from  an  ROC 
curve. 

Any  distribution  function  on  a  real  axis  consists  of  two 
parts  the  jumps  in  the  distribution  function  corresponding  to  values 
that  have  probability,  and  the  smooth  continuous  parts  of  the  distribution 
corresponding  to  those  values  that  have  zero  probability  individually, 
but  which  have  a  probability  density  function.  For  notation  use  F  for 
distribution  function,  f  for  probability  density  function,  z(i)  for  a 
point  on  the  decision  axis  for  which  the  distribution  function  under  N 
or  SN  has  a  discontinuity.  The  magnitude  of  these  jumps  will  be 
denoted  by  co(z). 

z 

t?/  SN,  r°  SN,  ,  v  /  ,.v  SN,  ,0  00v 

F(zq  N  )  =  J  f(z  N  )  dz  +  l,  os  (z(i)  N  )  a.  e.  (2.  22) 

-co  |z(i)  <  zQ| 

The  above  equation  represents  the  distribution  function  for  the  condition 
N  if  the  lower  conditions  are  read  throughout,  and  the  distribution 
function  for  the  condition  SN  if  the  upper  conditions  are  used  throughout. 
As  is  true  in  all  general  distribution  work,  we  can  guarantee  the  exist¬ 
ence  of  the  probability  density  function  almost  everywhere,  but  not 
everywhere.  This  is  because  there  may  be  a  finite  or  countable  number 


Strictly  speaking,  this  may  fail  to  be  true  at  some  points.  However, 
the  total  set  of  points  at  which  it  fails  is  a  set  of  zero  probability.  Thus, 


of  points  at  which  the  probability  density  is  not  specified  without  affect¬ 
ing  the  value  of  the  distribution  function. 

Concentrate  on  those  values  of  the  decision  axis,  z,  that 
have  probability  under  either  condition  N  or  SN.  If  the  probability 
under  N  is  positive  but  the  point  is  a  set  of  SN- measure  zero  the  like¬ 
lihood  ratio  will  be  zero. 

u)|  z(i) i N ]  >  0,  (ufz(i)iSN]  =  0  ,  £f z(i)J  =  0  (2.23) 

If  the  converse  situation  holds  with  the  point  z(i)  having  N- measure 
zero,  then  the  likelihood  ratio  of  that  value  is  infinite. 

co f  z( i)  IN]  =  0,  co [ z(i) i SN ]  >  0,  Cfz(i)J  =oo  (2.24) 

In  the  situation  where  the  point  is  of  positive  probability  under  both  con¬ 
ditions  N  and  SN  the  likelihood  ratio  is  given  by  the  ratio  of  these  two 
probabilities. 

w|z(i)!N]>0,  co  [  z(i)iSN)  >  0,  £[z(i)j  =  (2.25) 

By  the  fundamental  theorem  proved  in  Section  1.  2,  the 
likelihood  ratio  of  the  logarithm  of  the  likelihood  ratio  is  the  exponen¬ 
tial  of  the  log  of  the  likelihood  ratio. 

f(z)  =  eZ  (1.54) 

Equation  1.  54  is  a  direct  result  of  the  fundamental  theorem.  It  is  not 
obtained  by  inverting  Eq.  2.  21.  The  likelihood  ratio  in  Eq.  2.  21  is  a 


function  which  maps  the  observation  space  into  the  reals.  The  likeli¬ 
hood  ratio  given  in  Eq.  1.  54  maps  the  reals  into  the  reals.  This  distinc¬ 
tion  is  the  reason  the  fundamental  theorem  had  to  be  established. 

From  Eq.  1.  54,  under  the  three  conditions  of  Eq.  2.  23 
-  Eq.  2.  25,  the  values  of  z(i)  can  be  determined  exactly  at  which  the 
likelihood  ratio  takes  on  the  values  zero  and  infinity.  When  the  likelihood 
ratio  is  zero,  the  decision  axis  value  must  be  minus  infinity,  and  similarly 
when  the  likelihood  ratio  is  infinite, the  decision  axis  value  must  be  plus 
infinity.  All  other  jumps  in  the  distribution  functions  must  be  common 
to  both  of  the  distribution  functions,  and  the  ratio  between  the  two  magni¬ 
tudes  of  the  jump  will  be  given  by  Eq.  1.  54. 


£[z(i)j  =  0  z(i)  =  -  oo 


£  j  z(i) !  -  oo  =$>  z(i)  =  +  oo 

z(i)  +  ±:c,  o>[z(i)|SNj  =  eZ^  w|  z(i)!N] 


(2.  26) 


We  will  retain  the  form  of  Eq.  2.  22  for  the  N  distribution  function  but 
will  utilize  Eq.  1.  54  and  2.  26  to  write  the  distribution  function  under  SN. 


F|Z()iSNj 


ZQ  ■  z(i) 

J  e  f(zlN)dz+  ),  e  u> l z(i) I N ) 
-x  |z(i)<  zQJ 


a.  e. 
(2.  27) 


In  Section  2.  2.  \  the  N  distribution  function  could  be  chosen 
quite  arbitrarily  and  the  SN  distribution  function  chosen  to  obtain  the 
desired  ROC  curve.  In  contrast,  the  SN  distribution  function  of  z 
is  determined  once  the  N  distribution  function  is  chosen. 
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Let  us  analyze  the  ROC  curve  for  a  given  N  distribution 
function  for  z.  Let  ft  stand  for  a  particular  cut  value  along  the  z-axis. 
Whenever  the  value  ft  is  one  for  which  the  distributions  have  jumped, 
representing  a  z-value  with  positive  probability  under  both  conditions,  let 
r  be  a  number  between  zero  and  one  inclusive.  The  simple  decision 
based  on  the  z  decision  axis  with  parameters  ft  and  r  will  plot  as 
the  point  (x,  v) 

oo 

x(/3,  r)  =  f  f(zlN)dz+  7  co^zCi) i  N )  +  rw(/3iN)  a.  e.  (2.28) 

ft  jz(i»  0} 

y(ft.r)  =  f  eZf(z|N)dz+  7,  eZ  1  w(z(i)|N)  +refjw(/3|N)  a.  e. 

ft  |z(i)>/3|  (2.29) 

The  point  moves  up  along  the  ROC  curve  with  increasing  r  and  moves 
down  the  ROC  curve  with  decreasing  r,  in  the  range  of  r  from  zero 
to  one. 
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The  derivative  of  the  probability  of  false  alarm  (Eq.  2.  28)  with  respect 
to  the  parameter  r  is  the  magnitude  of  the  jump  under  N.  The  deriva¬ 
tive  of  the  probability  of  detection  (Eq.  2.  29)  is  the  magnitude  of  the 
jump  under  SN. 


P  e  jz(i)}  ,  0  <  r  <  1 


dy(<3,  r) 

dr 


e^  w(/3IN) 


(2.  30) 


The  ratio  of  these  two  derivatives  is  the  slope  of  the  ROC  curve 


P  =  In  y'(x) 


(2.  31) 


where  the  prime  denotes  differentiation  with  respect  to  the  argument. 
For  any  portion  of  an  ROC  curve  where  the  slope  is  constant,  the 
natural  logarithm  of  this  slope  corresponds  to  the  cut  value  on  the  z- 
axis.  At  the  end  points  of  the  linear  segment  corresponding  to  the 
value  r  =  0  and  r  =  1  the  value  of  the  cut  on  the  z-axis  can  be  deter¬ 
mined  from  the  appropriate  one-sided  derivatives.  The  r  value  can 
also  be  determined  for  each  point  along  a  linear  segment  of  the  ROC 
curve,  since  r  will  increase  linearly  from  the  value  zero  at  the  left- 
hand  point  of  the  segment  to  one  at  the  right-hand  point  of  the  segment. 


r  =  0 
r  =  1 


P  =  In  y'(x  +  0) 
p  =  In  y'(x  -  0) 


(2.32) 


For  a  cut  at  a  z  value  which  occurs  with  probability 
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zero  under  both  conditions  N  and  SN,  the  parameter  r  is  irrelevant. 

A  change  in  ROC  point  corresponds  to  a  change  in  the  cut  value  ft. 

If  there  are  no  jumps  in  a  z- neighborhood  of  a  cut  (3, 

( 

one  may  differentiate  the  coordinates  of  the  ROC  curve  with  respect  to 
p.  From  Eqs.  2.  28  and  2.  29,  it  follows  directly  that 


/3*  jz(i)}  =  -f(z=/3IN)  a.  e. 


dy(j3,  r) 
dp 


f(z  =  pi  N)  a.  e. 


(2.33) 


As  before,  the  ratio  of  these  two  derivatives  relates  the  value  of  the 
decision  cut  on  the  z-accision  axis  to  the  slope  of  the  ROC  curve. 

=  dx  ’  ^  =  1,1  yT(x)  a-  e-  (2-  34) 


We  have  shown  that  when  the  ROC  curve  is  obtained  from 
the  decision  axis,  which  is  the  logarithm  of  the  likelihood  ratio  of 
observation,  that  the  curve  is  differentiable  almost  everywhere,  and  the 
logarithm  of  this  derivative  is  numerically  equal  to  the  cut  value  on  the 
decision  axis. 

Synthesis: 

It  was  really  unnecessary  to  hypothesize  that  the  decision 
axis  was  the  logartihm  of  the  likelihood  ratio  of  some  (physical)  observa¬ 
tion  space,  2)C.  It  was  merely  necessary  for  the  variable  z  to  be  the 
logarithm  of  the  likelihood  ratio  of  something  and,  since  it  is  the  loga¬ 
rithm  of  the  likelihood  ratio  of  itself, the  internal  consistency  would 
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have  been  sufficient  to  specify  the  problem.  That  particular  circle  was 
avoided  because  the  logic  may  have  been  questioned.  Since  the  logic  has 
now  been  established  by  examples  here  and  by  the  fundamental  theorem. 
Section  1.  2,  we  can  adopt  this  seemingly  circular  logic  from  now  on. 

If  we  are  given  an  ROC  curve  which  is  complete  and  con¬ 
vex,  we  can  hypothesize  the  existence  of  a  decision  axis  which  is  the 
logarithm  of  its  own  likelihood  ratio  and  determine  for  each  point  on 
the  ROC  the  corresponding  cut  value  on  this  decision  axis;  and,  if 
necessary,  the  value  of  the  randomizing  parameter  r.  If  the  value  of 
the  derivative  is  constant  over  some  portion  of  the  arc  of  the  ROC,  the 
horizontal  extent  of  this  arc  indicates  the  probability  cu(z|N),  and  if  a 
value  of  the  derivative  has  been  taken  on  only  once,  then  Eq.  2.  34 
relates  the  slope  to  the  corresponding  cut  value  and  Eq.  2.  33  gives  the 
probability  density  function,  f(z  i  N). 

It  has  been  shown  that  every  complete,  convex,  non¬ 
singular  ROC  curve  corresponds  one-to-one  with  a  particular  distribu¬ 
tion  function  for  z  under  the  condition  N.  It  was  also  shown  that  the 
distribution  for  z  under  N  completely  specified  the  distribution  for 
z  under  SN.  Even  more  can  be  shown,  since  Eq.  2.  26  can  be  inverted 
to  read 

w[z(i)iN]  =  e'z(i)  w[z(i)!SN]  (2.  35) 

and  the  similar  relation  for  probability  density  functions  is 

f(z!N)  =  e'Z  f(ziSN)  (2.  36) 


Therefore  the  distribution  of  z  under  SN  completely  specifies  the  dis¬ 
tribution  of  z  under  N.  It  has  therefore  been  shown  that  every  complete, 
convex,  nonsingular  ROC  curve  corresponds  one-to-one  with  a  particular 
distribution  function  for  z  under  the  condition  SN.  Both  correspond¬ 
ences  relate  to  the  same  model;  they  are  different  ways  of  describing 
a  sufficient  part  of  the  observation  statistics,  from  which  all  of  the 
remainder  of  the  model  may  be  obtained. 

Another  type  of  generalization  is  possible.  Any  class  of 
functions  that  is  in  one-to-one  correspondence  with  the  distribution  of 
z  under  N,  will  be  in  one-to-one  correspondence  with  the  set  of  com¬ 
plete,  convex,  and  nonsinguiar  ROC  curves. 

2.  3  ROC  Character,  n( z) 

The  ROC  character  ti(z)  introduced  in  this  section  is  the 
principal  function  that  will  be  used  in  the  organization  of  ROC  curves 
into  families.  The  only  property  of  the  ROC  character  and  its  several 
related  functions  discussed  in  this  section  is  the  one-to-one  relation  with 
the  distribution  functions  for  z. 

Definition:  ROC  character  7t(z). 

Whenever  the  logarithm  of  the  likelihood  ratio,  z,  pos¬ 
sesses  probability  density  functions,  the  ROC  character  is  defined  as 

tt(z)  =  v'f(z|SN)  f(ziN)  I z |  <  oo  (2.37) 


The  ROC  character  can  be  used  to  obtain  the  two  probability  densities 
used  in  its  definition.  Since  the  likelihood  ratio  of  z  is  ez,  that  is 
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z  f(zlSN) 
e  “  f(z|N) 


(2.  38) 


it  follows  that 


+.  5z  ,  . 
e  tt(z)  = 


VfUiSN)f(zlN)  =  f(ziSN) 


(2.  39) 


In  a  similar  manner 


e'-5Ztr(z)  =  y  Vf(zlSN)  f(z  j  =  f(zlN)  (2.40) 


These  are  summarized  in  the  double  notation  equation 


±.  5z  ,  .  ,,  SN. 

e  ir(z)  =  f(z  N  ) 


(2-  41) 


When  the  distribution  functions  for  z  contain  discontinuities,  the  jump 
functions  (discrete  probabilities)  are  also  needed  to  specify  the  distri¬ 
butions. 

Definition:  1  OC  character  jump  (z) 

Letu>(z!SN)  and  cl>(z|N)  be  the  jump  functions  for  the 
distribution  of  z.  The  ROC  character  jump  is  defined  as 


GD  (z)  =  Vgd(zISN)  cc(ziN) 

71 


z!  <  x  (2.42) 


Utilizing  Eq.  2.  26,  it  follows  that 


±.  5z  ,  .  ,  SN. 

e  u’^z)  =  «(?  N  ) 


(2.  43) 
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The  ROC  character  is  obtained  from  probability  densities  and,  as  will 
be  formally  established  in  Section  3.  2,  has  many  of  the  properties  of 
a  probability  density.  The  function  that  is  analogous  to  a  probability 
distribution  function  is  the  integrated  ROC  character  Il(z). 

Definition:  Integrated  ROC  character  II  (z) 


n(zQ)  =  f  n(z)  dz  +  Yj  ^(z) 

“  OC  Z<Zq 


(2-44) 


The  integrated  ROC  character  will  be  useful  in  Stieltjes  integration, 
where  it  will  appear  written  dfl  (z).  As  a  memory  aid,  one  may  think 


of  Eq.  2.  44  as  reading 


memory  aid:  dll  fz)  =  r;(z)  dz  +  co^(z) 


(2.  45) 


The  distribution  functions  for  z  can  be  expanded  in  density  and  jump 


function  form. 


F(z  ^)  = 
{  0  N  ' 


Using  2.  41  and  2.  44 


f  f(z|SNN)dz+  V  u.(z|SnN)  (2.46) 


z<z0 


F(z  ^N)  = 
r(Z0  N ' 


j  e  ’  z  ti(z)  dz  +  Y  e±  Z  w  (z) 

J  L-J  Jf 

-=0  z<z0 


(2.  47) 


The  precise  relation  of  these  distribution  functions  to  the  integrated 


I  p 

1  B 


ROC  character  is  given  by  Eq.  2.  48 
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SN.  _  Zr  Z0 

N  ’  J 

z  =  -00 


±.  5z  ,IT ,  , 
e  dll(z; 


(2.  48) 


Although  almost  all  of  this  work  will  be  devoted  to  the 
distributions  of  z  (meaning,  as  always  herein,  the  logarithm  of  the  like¬ 
lihood  ratio),  it  will  be  convenient  to  have  a  function  for  arbitrary  prob¬ 
ability  density  functions  that  parallels  the  ROC  character. 

Definition:  Root  product  density  function  f  (  ) 

If  t  is  a  random  variable  with  probability  density  func¬ 
tions  f(tiSN)  and  f(t!N),  then  the  root  product  density  function  for  t 
is  defined  as 

f  (t)  =  J~\ f(t  ISN1  f(tiN)  (2.  49) 

7T ' 

2.  4.  Remarks  on  the  Distribution  of  z 

This  section  contains  four  short  remarks  about  the  dis¬ 
tribution  of  z  and  their  correspondingly  short  proofs.  The  first  three 
have  to  do  with  the  expected  value  of  z  under  the  two  conditions  N 
and  SN. 

Remark:  E(ziN)<0 
Proof: 

(1)  E(t  !N)  =  /  CdF(f  !  N) 

=  /  ((£)  dF(f  IN)  (2.50) 

=  /  dF(flSN)  =  1 


(2)  In  is  a  convex  function,  and  hence  lies  on  or  below  a 
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tangent 

z  <z  (1)  +  z'(l)(£-l)  =  £-1 
E(zlN)  <  E(£-11N)  =  0 

Remark:  E(  z  iSN)  >  In  E(£  iSN)  (2.51) 

Proof: 

In  is  a  convex  function 

Remark:  E(z  ISN)  >  0  (2.52) 

Proof: 

Consider  g(f)  =  f  In  C;  g(l)  =  0 
g ’(£)  =  1  +  In  £;  g'(l)  =  1 

g'V-)  =  r1;  g">  0 

g(£)  is  "convex  upward"  and  lies  on  or  above  any  tangent 
.-.  g(f)>g(l)  +  g'(l)(£-l) 
t  In  f  >  f-1 

E(f  In  f  N)  >  E(f-1N)  =  0 
E(?  In  « IN)  =  E(ln  C|SN) 

/.  E(ln  f  ISN)  2:  0  Q.  E.  D. 

Whenever  there  are  no  jumps  in  the  probability  distribu¬ 
tion  functions  for  z,  the  RJC  character  contains  all  of  the  information, 
and  there  is  no  need  to  use  the  integrated  ROC  character.  In  that  case, 
we  can  say  something  about  the  symmetry  of  the  ROC  character. 
Remark:  ,;(z)  is  symmetric  iff  it  is  symmetric  about  z  -  0 
Proof: 


Assume  ir(z)  =  jt(c  -  z) 


r 
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f(z 

IN) 

00 

dz  =  f  e 

'  5z  7r(z)  dz 

-oo 

-00 

00 
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5z 

tr(c-z)  dz  = 
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e  J  e 
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-00 

-  00 
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5c 

oo 

/ 
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=  e  l 
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■  oo 

-oo 

5c 

-  e 
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CHAPTER  m 


ROC  FAMILIES 

3 .1  Philosophical  Orientation 

The  objective  of  the  present  work  is  to  organize  ROC 
curves  into  families.  To  do  this,  some  restriction  has  to  be  made  on 
the  system  of  ROC  curves  to  be  considered.  The  first  intent  is  to 
treat  as  large  a  system  of  ROC  curves  as  possible.  The  second  intent 
is  to  develop  the  ROC  families  which  have  some  generality  while  still 
requiring  that  each  curve  in  the  family  can  be  specified  by  only  one  or 
two  parameters.  In  this  way.  the  determination  of  an  ROC  curve  can 
be  broken  into  a  two-step  process  of  determining  the  family,  and  then 
determining  the  one  or  two  parameters  necessary  to  specify  the  par¬ 
ticular  curve. 

In  this  work  I  have  been  influenced  by  Karl  Pearson, 
who  wished  to  unify  and  codify  the  treatment  of  probability  density 
functions  of  one  real  variable.  Restricting  himself  to  the  class  of 
unimodal  functions  he  worked  with  those  probability  densities  de¬ 
scribed  by  the  range  of  the  random  variable  and  the  first  four  moments. 
His  results  were  strongly  related  to  the  function  which  has  come  to  be 
known  as  the  Pearson  ratio.  For  a  probability  density  function  f(t), 
the  Pearson  ratio  is 

Pearson  ratio  =  ^  (In  f(t))  =  (3.  1) 
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Whenever  the  Pearson  ratio  is  a  rational  function  of  degree  no  higher 
than  a  monomial  over  a  quadratic,  the  density  function  is  within  the 
Pearson  system.  The  specific  type  is  based  on  the  roots  of  the  quad¬ 
ratic.  The  root  of  the  numerator,  the  monomial,  is  the  position  of 
the  mode  of  the  density  function.  Pearson  later  expanded  this  work 
oy  the  consideration  of  a  quadratic  over  a  cubic,  which  allowed  in¬ 
clusion  of  more  complicated  unimodal  functions  as  well  as  bimodal 
functions.  The  Pearson  ratio  plays  two  roles  of  special  interest  to 
this  present  work.  First,  the  Pearson  ratio  defines  the  area  of  appli¬ 
cability  of  the  Pearson  classification  system.  That  is,  the  mathema¬ 
tician  dealing  with  a  given  probability  density  function  could  decide 
definitely  whether  that  function  falls  within  the  Pearson  system  or 
not.  Secondly,  within  the  system  the  ratio  provides  a  classification 
into  families. 

In  attempting  to  codify  ROC  curves  the  present  work  is 
restricted  to  complete  convex  nonsingular  ROC  curves.  The  formal 
function  that  is  proposed  as  a  basis  for  classification  is  the  ROC  char¬ 
acter. 

The  classification  of  families  of  ROC  curves  would  be 
a  huge  undertaking  if  it  were  not  true  that  a  link  can  be  found  that  re¬ 
lates  this  classification  to  our  heritage  in  probability  theory  and 
statistics.  In  a  moment  the  relation  of  the  ROC  character  to  the 
rearson  ratio  will  be  determined  to  see  if  the  Pearson  system  can  be 
used  directly  for  classifying  ROC  characters.  In  the  next  section. 
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3.  2.  the  relationship  of  the  ROC  character  to  all  univariate  proba¬ 
bility  density  functions  is  obtained.  This  second  link  provides  the 
basis  for  the  classification  of  ROC  families  by  ROC  characters. 

The  relationship  of  the  ROC  character,  tt(z),  to  the 
Pearson  classes  is  obtained  from  the  relationship  of  the  SN  and  N 
density  functions  to  the  ROC  character. 


Divide  through  by  the  exponential 

/v  T.  5z  .  /  |Sn\ 
n(z)  =  e  f  (  z  |  N  ) 

and  take  natural  logarithms. 


(2.41) 


(3.2) 


In  tt(z)  =  lnf^zj^y+.5z  (3.3) 

The  functions  are  in  convenient  form  to  take  the  derivative  with  respect 
to  the  variable,  z. 


The  Pearson  ratio  for  the  ROC  character  differs  from  the  Pearson 
ratio  for  either  density  function  by  either  plus  or  minus  one-half.  It 
is  not  enough,  however,  that  each  of  these  Pearson  ratios  be  a  rational 
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function.  If  the  denominator  is  of  higher  order  than  the  numerator, 
then  the  inclusion  of  the  added  constant  in  obtaining  a  new  rational 
function  will  generally  lead  to  numerator  and  denominator  having  the 
same  order.  It  is  quite  common  to  find  that  the  distribution  for  the 
logarithm  of  the  likelihood  ratio  under  one  of  the  conditions  does  fall 
in  a  Pearson  class,  while  the  distribution  of  the  logarithm  of  the  likeli¬ 
hood  ratio  under  the  other  condition  falls  in  either  a  different  Pearson 
class  or  outside  the  Pearson  system.  Only  when  the  denominator  is 
of  equal  or  lower  order  than  the  numerator  will  the  two  distributions 
of  likelihood  ratio  and  the  ROC  character  stay  within  the  Pearson  sys¬ 
tem.  In  this  special  case,  all  three  will  be  in  the  same  Pearson 
class. 

The  Pearson  ratio  for  a  normal  probability  density  is 

Normal  Pearson  ratio  =  -m^n  -  (3.  5) 

variance 

Section  1.  3  contains  W.  W.  Peterson’s  result,  "When  the  logarithm 
of  the  likelihood  ratio  is  normally  distributed  under  one  condition,  it 
is  also  normally  distributed  under  the  other  condition.  "  We  can  add 
to  this  at  this  time,  by  adjoining  "and  the  ROC  character  will  be  pro¬ 
portional  to  a  normal  probability  density  function  and  centered  half¬ 
way  between  the  two  conditional  density  functions.  " 
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3.  2  Fundamental  Theorem  on  Regular  ROC  Characters 

In  this  section  it  will  be  shown  that  ROC  characters  for 
regular  ROC  curves  correspond  to  those  probability  distributions  on 
the  real  line  which  possess  moment  generating  functions.  Such  dis¬ 
tributions  have  received  considerable  study  in  the  history  of  probabil¬ 
ity  theory. 

All  probability  distributions  on  the  r  eal  line  possess 
characteristic  functions  (Fourier  transforms),  but  not  all  possess 
moment  generating  functions  (double  sided  LaPlace  transforms).  * 
Those  distributions  that  possess  both  will  have  finite  moments  of  all 
orders,  and  their  transforms  will  be  analytic  in  some  neighborhood 
of  the  origin. 

In  order  to  consider  both  discrete  probability  functions 
and  probability  density  functions,  we  work  with  probability  distribu¬ 
tion  functions,  denoted  by  F.  and  the  integrated  ROC  character,  n(z). 
For  discrete  random  variables  the  individual  probabilities  arc  the 
jumps'  in  the  distribution.  For  continuous  random  variables  the  proba¬ 
bility  density  function  is  the  derivative  of  the  distribution,  and  cor¬ 
respondingly  the  ROC  character  ff(z)  is  the  derivative  01  H(z). 

Fr  3m  Character  to  Distribution:  As  is  customary  in  probability  nota¬ 
tion  the  limiting  values  for  large  negative  and  large  positive  values 


pb::  example,  Ref.  30,  p.  12. 
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are  Indicated  by  the  arguments  -cc  and  +oc.  The  four  requirements 


on  the  integrated  ROC  character  for  regular  ROC  curves  are: 

(1)  the  lower  limit  is  zero  II(-oc)  =  0  (3.  S) 

(2)  monotone  growth  dll(z)  >  0  (3.  7) 

(3)  unit  "N17  value  /  e-'  5z  dn(z)  =  1  (3.  8) 

(4)  unit  "SN”  value  f  e+‘  ^z  dll(z)  =  1  (3.  9) 

Add  the  equalities  for  the  third  and  fourth  conditions  to  obtain 

/  [e' 5z  +  e"' 5z]  dn(z)  -  2  (3.10) 


5z  ***  Bz 

Since  the  sum  (e‘  +  e  '  '  )  is  greater  than  or  equal  to  2  for  all  z 

values. 

x 

n(+oc)  =  f  dH(z)  <  1  (3.  11) 

-x 

Let  c  be  reciprocal  of  IT(-kc). 

cn(+x)  =  1  (3.  12) 

This  has  provided  the  basis  for  the  distribution  function 
that  is  proportional  to  the  integrated  ROC  character,  and  the  proof 
that  the  corresponding  moment  generating  function  exists  in  some 
neighborhood  of  the  origin.  Consider  the  function 
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F(z)  =  cn(z)  (3.  13) 

From  (3.  6),  (3.  7)  and  (3.  13)  it  follows  that 


F(-oc)  =  0  F(+oc)  =  1  (3.  14) 

F  is  monotone  nondecreasing,  dF(z)  >  0 

Therefore  F  defined  by  (3.  13)  is  a  distribution  function. 

The  definition  of  the  moment  generating  function  is  usu¬ 
ally  given  using  the  expected  value  operator,  E(  ). 


Mp(S)  =  E(e^) 

This  may  be  evaluated  using  the  distribution  function 


(3.  15) 


toF(£)  =  /  e^dF(t) 


(3.  16) 


If  the  distribution  function  contains  no  jumps,  the  density  function  may 
be  used. 


mf(0  = 


e^  f(t)  dt 


(3.  17) 


Mp(0  is  called  the  moment  generating  function  because 


dnMF(|) 


=  E(tn) 


1=0 


(3.  18) 
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Only  those  random  variables  which  are  sufficiently  "concentrated” 
will  possess  a  moment  generating  function  and  finite  moments  of  all 
orders. 

A  simple  dominance  method  can  be  used  to  show  that 
the  moment  generating  function  exists  near  the  origin  £  =  0.  For  any 
real  £  with  absolute  value  not  greater  than  one- half. 

MfU)  =  /  e^z  dF(z)  (3.19) 

-oc 

OC  o 

=  cf  e^z  dll(z)  (3.20) 

-oc 

00  t  t 

<cf  e4z  +  e"4z  dll(z)  (3.21) 

-oc 

c  /*  e‘  5z  +  e'-  5z  dn(z)  -  2c  (3.  22) 

-oc 

From  Distribution  to  Character.  To  satisfy  the  requirements  on  the 
integrated  ROC  character,  specifically  the  unit  N  and  unit  SN  values 
listed  in  Eqs.  3.  8  and  3.  9,  n(z)  may  differ  from  a  parent  distribu¬ 
tion  by  a  shift  and  a  scale  factor.  Start  with  any  probability  distribu¬ 
tion  function,  F(t),  which  possesses  a  moment  generating  function, 
Mj,(£).  Later,  values  will  be  specified  for  the  constants  a,  b,  and 
c,  the  latter  two  positive.  For  the  moment  consider 


z 


a  +  bt 


(3.  23) 
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The  moment  generating  function  for  the  random  variable  z  is 


Eje^z]  =  E[e^(a+bt)3  =  ea^  Mp(b^) 

(3.24) 

that  is 

) 

/  e|z  dF(^S)  =  ea?  MF(b?) 

(3.25) 

The  specific  values  of  interest  are 

/  e*  •  52  dF  (zb  a)  =  e*  •  5a  Mp(± .  5b) 

(3.  26) 

For  any  positive  value  of  b  which  is  sufficiently  small 

function  Mj,(±  .  5b)  exists,  set 

so  that  the  j 

a  =  In  Mf(-  .  5b)  -  In  MF(+.  5b) 

(3.  27) 

c  -  [MF(-.5b)MF(+.5b)]'5 

(3.  28) 

The  decimal  exponent  .  5.  the  fractional  exponent  1/2.  and  the  square 
root  symbol  all  mean  the  same  thing.  The  decimal  exponent  is  con¬ 
venient  for  the  type  of  manipulations  in  this  present  work.  If  the 
constants  have  been  chosen  correctly,  then  the  following  is  an  integra¬ 
te  d  ROC  character 


n(z)  =  c'1  F  (~i“) 


(3.  29) 
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If  the  distribution  function  F  is  the  integral  of  a  probability  density 
function,  f.  then 

ir(  z)  =  (bc)-1f(^~ )  (3.30) 

The  final  step  is  to  determine  that  Eqs.  3.  8  and  3.  9  hold. 
From  <3.  26)  and  (3.  27), 

/  e*-5zdF(Z-^.)  =  [MF(-.5b)]±'5[MF(+.5b)]:f-5MF;±.5b)  (3.31) 

/  e*  5zdF<rii)  [MF(-.5b)MF(+.  5b) j  ■  5  =  c  (3.32) 

From  (3.29), 

f  e±-  5z  dll(7)  -  c"1/  e±-5zdF(^)  (3.33) 

From  (3.32)  and  (3.331, 

/e±'5zdn(z)  =  o^c  =  1  (3.34) 

There  is  a  continuous  range  of  b  values.  Correspond¬ 
ing  to  each  b  value  are  appropriate  constants  a  and  c.  Using 
these  values,  the  simple  scaling,  Eq.  3.  23,  generates  a  family  of 
(integrated)  ROC  characters. 

The  key  relations  are  repeated  for  later  reference. 

Given  a  distribution  function  F(t),  or  a  moment  gener¬ 
ating  function  Mp(£).  for  any  positive  value  of  b  sufficiently  small 
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that  Mp(±.  5b)  exists,  let 

a  =  lnMF(-.5b)  -  lnMF(+.5b)  (3.27) 

c  =  [Mf(-  .  5b)  Mf(+.  5b)]  •  5  (3.  28) 

Then 

n(z)  =  c_1  F(^)  (3.29) 

tr(z)  =  (bcf1  f(~p)  (3.30) 

are  valid  integrated  ROC  character,  and  ROC  character,  respectively. 

3.  3 _ Classical  ROC  Characters 

The  development  of  a  system  of  ROC  families  will  begin 
with  probability  density  functions  of  a  real  variable  which  have  moment 
generating  functions.  Each  such  probability  density  function  corres¬ 
ponds  to  some  ROC  character.  If  the  probability  density  function  is 
well  known  and  has  a  universally  accepted  name,  that  same  name  will 
be  used  for  both  the  ROC  character  and  the  resultant  ROC  family  of 
curves.  Certain  classes  of  probability  density  function  are  known 
by  several  different  names,  because  they  were  independently  developed 
in  several  fields  of  applications.  An  example  is  the  normal,  Gaussian, 
or  error  function.  An  attempt  has  been  made  to  relate  this  present 
work  to  each  of  the  names  used  in  the  literature. 

Armed  with  a  table  of  probability  density  functions  and 
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their  moment  generating  functions,  one  may  mechanically  generate 
classes  of  ROC  characters.  The  ROC  curves  are  found  by  direct 
integration.  The  basic  equations  for  the  ROC  curve  in  terms  of  the 
z-axis  cut  value  j3,  and  the  randomizing  parameter  r  are  given  by 
Eqs.  2.  28  and  2.  29.  When  /3  is  a  point  of  continuity,  or  otherwise 
when  r  =  0,  the  ROC  curve  equations  can  be  written  compactly  using 

°c  L  p. 

/  e1,ozdn(z)  (3.35) 

0 

The  following  sections  list  the  results  for  the  majoi' 
classical  probability  density  functions.  Some  mechanical  comments 
are  in  order  before  proceeding  with  these. 

(1)  The  fundamental  theorem  of  ROC  characters  allows  for  a  transla¬ 
tion  and  scaling  of  the  variable. 

z  =  a  +  bt  (3.23) 

An  entire  probability  density  function  class  may  usually  be  considered 
by  using  the  normalized  form  for  the  density  function,  having  zero 
mean  and  unit  variance,  since  Eq.  3.  23  rescales  the  mean  and  the  var¬ 
iance. 

(2)  Bounded  probability  density  functions  on  a  bounded  range  will 
always  have  moment  generating  functions.  Many  of  these  correspond 


the  form  of  Eq.  2.  48. 

Y  ‘(0) 
X(0) 


to  sections  of  probability  density  functions  with  infinite  or  semi¬ 
infinite  range.  Section  3.  4  will  treat  these  separately.  The  beta  and 
rectangular  density  functions  are  not  sections  of  other  probability 
density  functions,  and  are  included  in  the  following  sections. 

(3)  The  table  upon  which  the  following  sections  are  based  is  Table 
26.  1  of  Ref.  6. 

3.3.1  Normal,  Gaussian,  Error  Function.  For 

-CO  <  t  <  00 

.  li  i2 

f(t)  =  —  e  2  ,  M vtt)  = 

VSF  F 

Pi 

M-Ji  .  5b)  =  e  ®  exist  for  all  b  >  0 

r 


b2 

T 


_  b2  _  |  _  u2 

n(z)  =  e  <&(r)  =  e  ®  J  — e  du 

-oc  V2iT 
and 
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tt(  z) 


_1 

VT-  .  25b2  /  1  -  .  5b\  b 
b  \  1  +  .  5b  / 


z 


z  >  In 


If  we  let 


b 


2 


A-  1 
A+l 


that  is,  let 


A 


1  +  .  5b 
1  -  .  5b 


A  >  1 


then 

_ 1_  z  A  +  l 

v  (z)  =  (A-l)  1A^1e^^1,  z>  -In  A 
or 


tt(z) 


A  1  Z  -  Z  „ 

r  A+l  o 
A' 5  "A-l  2 
7A-H  e 


z  >  z  =  -In  A 
o 


The  ROC  curve  is 


X  = 


Y 


A 


3.3.3  LaPlace,  Double  Exponential.  For  -oc  <  t  <  oc 

f(t)  =  -g  e'lt!  Mf(4)  =  (i-  I2)’1 

MF(±.5b)  =  (1  -  .25b2)"1 


exists  for  0  <  b  <  2 


c  =  (1  -  .25b2)'1 

Iz  1 

*5 F  K  b 

_  Izl 

»(z)  =  — 2b25b"  e  b 

If  we  let 

b  ■  2L! 

that  is,  let 

0  *  2-+b  0  >  1 

then 

izl  B+l 

z(z)  =  B  e  --T  B-  1 

b2  •  : 

The  ROC  curve 

is 

X  -  (0+1>B'1  y0 

bb 

from  (0, 0)  to  the  negative  diagonal,  and 
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from  the  negative  diagonal  to  (1, 1).  The  negative  diagonal  point  is 


X  = 


B  +  1 


Y  = 


B 


B+  1 


3.3.4  Pearson  Type  III,  Gamma,  Chi-Square.  For 


t  >  0,p  >  -1 


f(t)  =  tp  e_t/r(p+l) 


MpU)  =  (1-5)' 


■(p+1) 


MF(±.5b)  =•  (l±.5b) 


•(P+1) 


a  =  (p+1>  ln(rHI) 

c2  =  (!  -  .25b2)_(/Jfl) 


so 


7T(z) 


(1  -  ■ 25ba) 

‘  b0+1  r  (p 


£±1 

2 


exists  for  0  <  b  <  2 


~f  [z  - (ptl) 


P  " 


z-a 


z  >  a 


As  before,  let 


B  =  B  >  1 


ff(z)  = 


/B  "+1 


z-aB  +  1 
2  B^T 


B-  1 


TJp+i)  (z  +  <'><a)  ln  B)P 

z  >  -(p+1)  In  B 
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or 


Jr(z)  = 


B>  5(p+l) 

(B-i)  r(p+i) 


(»)■ 


B+l  Z~Zo 
B-  1  2 


z  >  z  =  -(p+1)  In  B 


3.3.5  Fisher-Tippett  Type  I.  For  -oc  <  t  <  oc 


F(t)  =  e“e  M„(0  =  r(l-£) 


-t 


and 


f(t)  =  e 


-t-e 


-t 


MJ±  .  5b)  =  T(1  t  .  5b)  exists  for  0  <  b  <  2 

r 


a  -  In  HI  +  .  5b)  -  In  T{  1  -  .  5b) 


c2  =  T(1  f  .5b)  T(1  -  .5b) 


rr(z)  = 


b  v^r(l+.  5b)  (1-.  5b) 


r(l+.  5b) 


r(l-.5b) 


i  z  f  r(i+.  5b)  1 

e^Lr(1-5b)J 


z 

b 


m,  n  >  1 


3. 3.  6  Beta,  Pearson  I  and  XII.  For  0  <  t  <  1; 


Mpd)  =  M(m.  m  +  n,  §) 


,  .  ,  .... _ -  ,1  -/■  -  ’  V» 

STT? .  ■  ■■_  ■■  t  ~  ^ 
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This  is  Kummers  confluent  hypergeometric  function,  and  yields  a 
power  series  form  for  all  b  for  positive  n.  m 


m„(±  .  5b)  =  V  (±i)k  b 

F  k=0  -  (m+n+k-1)!  (m-l)I  2k  tCf 


This  is  so  unwieldy  that  the  general  form  is  retained 

,  _  .  m- 1  o  .  n- 1 

.(z!  =  (bcB(m,n))'1(^)  (2±[p.)  ,  a<z<b 

where 


a  =  In  Mp(- .  5b)  -In  Mp(+.  5b) 
c  =  [MF(-.5b)MF(+.5b)]-5 

For  m  =  1  or  n=  1  the  infinite  series  becomes  a  finite  sum,  but  no 
real  simplification  results.  ROC  curve  formulae  have  not  been  obtained. 
3.3.7  Rectangular.  For  0  <  t  <  1 

f(t)  =  1 

Mf(|)  =  J(e^  -  1) 

Mp({)  =  (± .  5b)  =  ±  2  b' 1  (e*  *  5b  -  1) 
a  =  - .  5b 

.25b  25b . 

~  e  ) 


be  =(2  e 


If  we  let 


b  =  2  ln(l  +  R)  R  >  1 


be  =  2R/VT+R 


n(z)  -  ,  izi  <  ln(l+R) 


The  ROC  curve  is  a  symmetric  rectangular  hyperbola 


(1  +  Rx)  (1  +  R  -  Ry)  =  1  *  R  R  >  1 


The  asymptotes  lie  outside  the  ROC  unit  square  at  a  distance  R 
3.3.8  Geometric.  For  integer  t>0;0<q<  1 


«>(t)  =  (l-q)qt 
M(0  =  (1-  q)/(  1  -  qe^ ) 


M(±.5b)  =  (l-q)/(l-qe±*5b)  .  0  <  b  <  2ilnq 


(r 


.  DD  \ 

_ \ 

-.5b) 

-  qe  / 


c  =  (l-q)/tl  +  qa-q(e'5b  +  e"-5b))' 

-i 

(z)  =  c  (l-q)q  ,  z  =  a.  a  +  b,  a  +  2b,  . . 
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w  (z) 

IT 


=  (l  +  q! 


qe 


5b  - .  5b 

-qe 


.  5/  .  .  5b  \ 

1 

\l-qe  / 


UsslI 

I  b  i 


JMi 

b  i 


write 


In 


( 


1  -  qe. 


1  -  qe 


,  5b  \ 

75b  ) 


and 


z^  —  z q  +  kb  y  k  -  0, 1, 2,  ... 


Let 


A  = 

The  ROC  curve  subscribes 


In  q  -  .5b 
lnq  +  .5b 

X  =  and  touches  it  at 


k  =  0, 1, 2, 


3.3.9  Binomial.  For  integer  t  >  0;  G  <  p  <  1 


w(t)  =  C”  p^l-p)”'1 

where 

n  n! 

C"  =  binomial  coefficients  =  ^  j(n-t) ! 


u>(t)  =  (1  +  e  )  C“  e 


M(?)  =  (1  +  eu)  (eu+l  f  1) 


u- .  5b  , 

,  e  4  1 

n  in - ?r - 

u+.  5b  * 
e  +1 


(eu  +  e-5b) 
(eu+-  5b+l) 


-  .  5bn 


urn/  2u  u-.5b  lNl5n 
(1  +  e  )  (e  +  e  +1) 


c  =  (i  +  eu)  n  2n  e'  5un(.  5  cosh  u  +  .  5  cosh  .  5b)'  5n 


0)  (z)  =  C  1  CL)  (t  =  ~^) 

7T  D 


=  [eu(.  5  cosh  u  +  .  5  cosh  .  5b)]"'  5n  2-n  c“  eut  ,  z  =  a  +  t 


The  term  2  c"  has  a  ’’bell- shaped"  graph  and  sums  over  t  to 
unity.  The  symmetric  case  of  p  =  .  5,  oi  u=  0,  yields  a  =  -  .  5bn 


&>  (z )  =  [1  +  .  5  cosh  .  5b]"'  5n  2n  c"  ,  z  =  b(t-  .  5n) 


The  ROC  curve  consists  of  n  +  1  straight  segments,  with  slope 
change  of  eD  at  each  vertex.  The  equation  of  the  ROC  must  be  ob¬ 


tained  from  the  two  parameter  tabi '  of  binomial  sums. 
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3.3,10  Specific  Binomial. 


n  =  0,  one  point:  from  the  general  binomial 
a  =  0,  c  =  1,  and  t  =  0 

ct>(z)  =  c  1  co (t)  =  1  and  z  =  a  =  0 
This  is  the  chance  diagonal,  Y  =  X. 


n  -  1,  two  points 

,  x  r  2u  u+5b  u-.5b  J"'5 
u>(  Zq)  =  e  +e  +  e  +1 


.  5b  u 

zi  =  ln-5T5btiLTr  u(zi> 
e  +  e 


eU  a>(z0) 


The  RCX:  curve  is  called  a  MLuce-ROC"  after  Duncan  Luce.  (Ref.  35) 
Z1 

Y  =  e  X  from  (0, 0)  to 

z0 

Y  =  1-e  (1-X)  from  the  vertex  to  (1, 1) 

For  this  special  case  it  is  easier  to  write 

ROC  Curve  is  line  segment  from  (0,0)  to  (Xq,  Yq) 

from  (X0,Y0)to  (1,1) 


and 
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.  Ail? 

z0  "  ln  1  -  X, 


o.(zJ  =  V(l-  Yft)(l  -  xT) 


Z1  =  ln  X 


0 


w(Zl)  =  Vy0X 


0 


3.3.11  Negative  Binomial,  Pascal,  PoTya.  For  in¬ 
teger  t  >  0;  0  <  p  <  1,  and  q  -  1-p. 


+’(t)  =  C^"1  P^'P^ 


MU)  =  pr  e*'  5br  (1  -  qe*-  5b)  0  <  b  <  2llnqi 


a  r  ln 


-.5b 

_ UL 

*  5b 

e  -  q 


r  .  2  +.  5b  -.5b  _ 

c  =  p  1  +  q-qe  -qe  -.5r 


“*<2)  =  ( 


.  2  .5b  - .  5b 

1+q-qe  -  qe 


)'5r  c^-'nV 


t  =  0.  1,  2.  ... 


and 


z  =  a  +  bt 

The  extensive  use  of  tables  is  necessary  to  plot  the  ROC  curves. 

3.3.12  Poisson.  For  integer  t  >  0;  m  >  0. 


u>(t)  = 


-m  t 
e  m 

t! 
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M(|)  =  em(e5  '  !) 

,  ±  5b 

M(±  .  5b)  =  em<e  * 
a  =  -2m  sinh  .  5b 
2m[sinh  .  25b]2 

C  -  6 

,  \  -  m  cosh  .5b  t n  i  n 

(j)  (z)  =  e  m  /t !  t  =  0,  1,  2,  . . . 

tr 

z  -  bt  -  2m  sinh  .  5b 

The  ROC  curves  can  be  obtained  from  chi-square  tables,  using  the 
identities 


x 

V 

k 


(me*-5*) 

t! 


=  P(X2  =  2me±-  5b!^=2k) 


Many  of  the  classical  distribution  functions  for  continu¬ 
ous  and  discrete  variables  do  not  possess  moment  generating  functions. 
Among  these  are  the  Cauchy  distribution  and  generalizations  of  the 
Cauchy  distribution,  the  Pearson  Types  IV,  VI,  and  XI,  the  F  dis¬ 
tribution  and  the  Student’s  t  distribution.  These  cover  a  semi-infinite 
or  an  infinite  range,  and  have  "tails”  which  go  to  zero  as  some  power 


of  the  random  variable. 
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3.  4  Truncated  ROC  Characters  and  the  Metastatic  Transformation 

ROC  curves  which  approach  the  origin  at  some  slope 
other  than  infinity  or  approach  the  point  (1, 1)  at  some  slope  greater 
than  zero  correspond  to  ROC  characters  whose  z- ranges  are  bounded 
away  from  +oo  or  -oo,  respectively.  There  is  a  functional  relation¬ 
ship  between  many  of  these  characters  and  those  corresponding  to 
ROC  curves  covering  the  complete  range  of  slope  from  infinity  to  zero. 
There  is  similarly  a  geometric  relation  between  the  two  types  of  ROC 
curves.  This  correspondence  shall  be  called  the  metastatic  relation. 
The  word  comes  from  the  medical  term  metastasis,  meaning  a  por¬ 
tion  of  something  which  has  broken  off  and  grows  in  a  new  place,  yet 
retains  its  original  character. 

The  metastatic  transformation  will  be  used  in  Section 
3.  5  to  relate  each  nonregular  but  complete  and  convex,  ROC  curve 
and  character  to  a  regular  ROC  curve  and  character.  This  will  extend 
the  classification  system  based  on  associated  distributions  to  these 
nonregular  ROC. 

3. 4.  1  The  Transformation.  The  algebra  of  the  trans¬ 
formation,  and  the  geometric  interpretation  will  be  discussed  at  each 
step  in  the  following  development.  Figure  3.  1  displays  a  particular 
regc’ar  ROC  curve;  Eq.  3. 36  is  a  formal  representation  of  the  cor- 

t 

responding  functional  relationship. 


TJ5~~ . 


First  ROC  curve:  Y^  =  Y^(X  ) 


(3.  36) 


Y,(b) 


Yj(a) 


Fig*  3. 1.  A  particular  ROC 


curve 


Also  sketched  in  Fig.  3. 1  are  the  coordinate  lines  at 

X‘  =  *’  Y1  =  YlW-  yi  -  *,<■».  These  ta  a  rectangle 

w.ih  the  firs.  ROC  curve  passing  through  the  lower  left  and  upper 

curve  is  interior  to  the  rectangle,  because  the  firs,  ROC  curve  is 
convex. 

The  metastatic  transformation  is  the  mapping  0f  this 
rectangle  onto  a  uni,  ROC  square.  Geometrically,  the  recta*, e  is 
removed,  and  the  coordinate  axes  uniformly  stretched  to  extend  from 

2610 1°  °ne'  The  tranSformed  lrc  ot  ‘he  first  ROC  curve  is  the  sec¬ 
ond  ROC  curve,  the  metastatic  image.  Is  the  second  ROC  regular? 

Yes,  it  is  complete,  convex,  and  inter, orto  the  unit  ROC  square 
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except  at  (0, 0)  and  (1,1).  An  equation  for  the  new  ROC  curve,  (X2,  Yg), 


X2  =  (Xra)/(b-a) 

Y2  =  (Y^Xj)  -  Yj(a ))/6fj(b)  -  Yt(a)) 


a  <  Xj  <  b 


(3.37) 


Z1 

When  a  point  (X^,  Yj)  with  slope  e  maps  onto  a  point  (X2,  Yg)  the 
slope  is  multiplied  by  the  ratio  of  the  Y  to  X  axis  expansion.  For¬ 
mally 


z2  dY2  b-a  Z1 

6  “  dX2  ‘  Yj(b)  -  Yj(a)  G 


(3.  38) 


Therefore 


Z2  =  Z1  +  ln  Yj(b)b-  Yx(a) 


(3.  39) 


The  second  z-axis  is  simply  a  translation  or  horizontal  shift  oi  the 


first  z-axis. 


The  ROC  character  for  the  second  ROC  curve  can  be 


obtained  as  follows.  The  formal  relationship  between  the  ROC  charac¬ 
ter  and  the  probability  density  function  of  the  log  likelihood  ratio,  con¬ 


ditional  to  the  condition  SN  is 


»2(z2}  =  e 


f(z2!SN) 


(3.  40) 
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This  probability  density  function  can  be  found  by  differentiating  the 
corresponding  distribution  function,  1-  Yg- 

f(z2ISN)  -  ^  (1-Y2) 

d(l-  Yn)  d(l-YJ  dz, 

=  _ _ L  _ I  \  (3 

dd-Yj)  dzj  dz2  |  1 

=  Y.(b)  -  Y.(a)  f(zlISN) 

1  1  J 

The  probability  density  function  of  the  first  log  likelihood  ratio  in 
terms  of  its  ROC  character  is 


fdjlSN) 


.  5z  j 
e 


From  Eq.  3.  39  which  relates  the  two  z-axes. 


.5z2 

e 


b-  a 

Y j(b)  -  Yj(a) 


.  5 


(3.42) 


(3.43) 


Equations  3.  40  through  3.  43  lead  to  the  equation  for  the  second  ROC 
character 


trj(z  f 


In 


Yj(b)  -  Yt(a) 


b-  a 


(b-a)  (Yj(b)  -  Yjfa))- 


(3.44) 


Let  us  draw  a  parallel  between  the  geometric  trans¬ 
formation  of  the  ROC  curves  and  the  algebraic  changes  in  the  ROC 
character.  Both  a  horizontal  and  vertical  scaling  convert  the  rec¬ 
tangle  to  the  unit  square.  In  Eq.  3.  44  there  is  a  vertical  scaling  given 
by  the  denominator,  and  a  horizontal  translation,  not  scaling,  given  by 
the  added  constant  in  the  argument  of  the  numerator.  The  second  par¬ 
allel  has  not  been  explicit.  Only  an  arc  of  the  original  ROC  curve 
was  transformed,  not  the  entire  curve.  The  bounds  on  the  Zg  axis 
correspond  to  the  upper  and  lower  bounds  on  the  z^  variable  at  the 
two  ends  of  the  arc  by  Eq.  3.  39.  In  summary,  the  ROC  curve  was 
cut.  and  a  small  portion  selected.  This  parallels  the  condition  that 
only  a  portion  of  the  original  character  is  used  by  the  metastatic  char¬ 
acter.  The  ROC  curve  was  re- scaled  to  fit  the  unit  square;  the  ROC 
character  has  been  re- scaled  according  to  its  necessary  properties. 

In  Fig.  3.  2  are  two  specific  ROC  curves  and  their  cor¬ 
responding  characters.  The  first  is  a  normal  ROC  curve  with  index 
d  of  2.  828.  Above  it  is  the  normal  ROC  character,  a  curve  propor¬ 
tional  to  the  normal  density  function,  centered  symmetrically  about 
zero  and  extending  from  -qc  to  +oc.  The  metastatic  transformation 
chosen  to  illustrate  the  process  was  the  part  of  the  arc  from  (0, 0)  to 
the  negative  diagonal.  This  corresponds  to  the  Zj  axis  from  zero  to 
infinity.  A  metastatic  transformation  maps  the  dotted  rectangle  to  the 
jntt  square.  The  transformation  on  the  ROC  character  truncates  the 


ROC  character  on  the  left,  re- scales  it  vertically  (the  scaling  is  a 
factor  of  2.  5  for  this  specific  example),  and  translates  the  axis  hori¬ 
zontally  the  appropriate  amount.  The  resulting  ROC  curve  and  char¬ 
acter  are  shown  on  the  right  in  Fig.  3.2.  The  ROC  curves  are 
repeated  in  Fig.  3.  3  on  normal- normal  probability  paper. 

Figure  3.  4  shows  ROC  curves  before  and  after  a  meta¬ 
static  transformation  from  normal  with  d  =  6.  55.  The  metastatic 
ROC  curve  is  the  image  of  the  original  ROC  curve  from  the  (0,  0) 
point  to  the  negative  diagonal.  Both  of  these  metastatic  "half- normal" 
ROC  curves  appear  to  be  binormal  ROC  curves  when  plotted  on  nor¬ 
mal-normal  paper. 

3.  4.  2  Self- Metastatic  Families.  Certain  families  of 
ROC  curves  are  self- metastatic;  that  is,  a  metastatic  transformation 
of  one  character  will  yield  a  second  character  which  is  in  the  same 
class.  The  most  startling  of  these  cases  is  the  pure  power  ROC 
curve  with  exponential  ROC  character  (see  Section  3.  3.  2).  This  is 
the  type  corresponding  to  the  ROC  curve 

x  =  yA  (3.45) 

A  metastatic  transformation  of  the  power  ROC  curve  containing  the 
point  (0. 0)  to  any  point  along  the  curve  is  the  same  as  the  original. 

The  formal  proof  is  in  Appendix  B. 

The  rectangular  hyperbola  ROC  curve  has  a  rectangular 
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(flat)  HOC  character.  Such  a  character,  being  a  simple  rectangle, 
is  symmetric  about  its  midpoint.  The  midpoint  must  be  z  =  0 
since  this  is  the  symmetry  point  for  all  symmetric  ROC  characters 
(see  Section  2.  4).  If  a  Hat  character  is  truncated  either  on  the  right 
or  on  the  left,  or  both,  and  then  the  translation  and  rescaling  dicta¬ 
ted  by  the  metastatic  transform  rules  performed,  the  character  will 
again  be  flat  and  symmetrically  placed  about  z  =  0.  Since  the  char¬ 
acter  will  not  span  as  much  of  the  z  axis  as  the  original  character 
did,  the  new  ROC  curve  will  be  different  from  the  first:  it  will  still 
be  a  rectangular  hyperbola.  The  more  that  is  removed  from  the  ori¬ 
ginal  character,  the  poorer  the  resulting  right  hyperbola  ROC  curve. 
As  a  family,  rectangular  ROC  are  self- metastatic. 

It  will  be  shown  in  Chapter  V  that  the  ROC  character 
for  the  conic  class  is 


-1.  5 

tt(z)  =  ±  K(Aez  +  2  B  +  Ce"z) 


(3.  46) 


where  the  range  of  the  log  likelihood  ratio,  z,  depends  on  the  specific 
curve  being  considered.  The  coefficients  A,  B,  and  C  are  the  first 
three  coefficients  in  the  classical  form  of  a  conic  section.  The  trans¬ 
lation  of  the  z  axis,  Eq.  3.39,  will  be  absorbed  into  the  new  coeffi¬ 
cients  A  and  C,  magnifying  one  and  diminishing  the  other  propor¬ 
tionately.  The  discriminant  for  a  conic  section  in  classic  form  is 
B2-AC.  Any  translation  of  z,  although  changing  the  coefficients  A 
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and  C  ,  will  leave  the  value  of  the  discriminant  unchanged.  Not  only 
will  the  individual  subtypes  of  conic  section  (hyperbola,  parabola, 
ellipse),  be  self- metastatic  families,  but  each  value  of  the  discrimi¬ 
nant  will  correspond  to  a  self- metastatic  subfamily. 

3.5  Nonsingular,  Nonregular  ROC 

The  only  difference  between  the  complete  convex  non¬ 
singular  HOC  curve  and  a  regular  ROC  curve  is  that  the  initial  left- 
hand  point  of  the  ROC  curve  may  begin  at  some  nonzero  value  and/or 
attain  unity  probability  of  detection  at  some  value  oi  false  alarm  proba¬ 
bility  less  than  one.  Such  a  complete  convex  nonsingular  ROC  curve 
is  illustrated  in  Fig.  3.  5.  It  should  be  emphasized  that  both  condi¬ 
tions  of  starting  above  zero  and  terminating  before  one  are  not  neces¬ 
sary,  either  one  is  sufficient  to  make  the  curve  nonregular.  The 
singular  ROC  curve  is  simply  the  upper  edge.  Nonsingular  means 
that  the  initial  point  is  not  as  high  as  one,  nor  is  the  value  of  false 
alarm  at  which  it  reaches  unity  detection  as  small  as  zero. 

Nonsingular  =J>  Y(Q)  *  1,  X(l)  t  0  (3.47) 

On  Fig.  3.  5,  a  metastatic  transformation  of  the  rectangle  from  Y(0) 
to  one,  and  from  zero  to  X(l)  would  produce  a  regular  metastatic 
image.  For  this  metastatic  image  the  ROC  character  would  be 


3.  5.  A  complete,  convex,  nonsingular  ROC 


(3.  48) 


irj(z) 


[X(1)(1-Y(0))]'5 


Consider  any  ROC  character,  it 2 (z)  with  either  bounded 
or  unbounded  range  denoted  by  z2  to  z2  • 

Given:  ff^z)  ^  z2  <  z  <  z2  (3. 49) 


If  this  were  the  metastatic  image  of  some  arc,  such  as  in  Fig.  3.  5, 
tb  :  ROC  character  for  the  regular  part  of  the  original  arc  would  be 


ffj(z)  =  [X(l)  (1  -  Y(0))] 


In 


1-Y(0)\ 


z 


2 


*  In 


1-Y(0) 

■xnr 


<  z  <  Zg  -  In 


1- Y(0) 

~wr 


(3.  50) 


The  complete  description  of  the  nonregular  ROC  curve  contains  the 
boundary  conditions. 

P(z  2  =  -oc  1 N)  =  l-X(l),  P(Zj  =  fociSN)  =  Y(0)  (3.51) 

Such  boundary  conditions  cannot  be  included  in  the  ROC  character 
since  probabilities  are  obtained  by  multiplying  the  character  value  by 
e± .  5z  ,p^s  muittpner  iS  either  zero  or  infinite  at  the  extreme  bound¬ 
ary  values. 

If  the  values  for  the  boundary  were  unknown,  they  could 
be  obtained  by  integrating  the  nonregular  ROC  character  over  the 

% 

.  f  v  ;/ 

t,  : 
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range  between  -oc  and  +oc. 


/  e+‘  5z  dn^z)  =  1  -  Y(0) 

-cC 

(3.  52) 

/  e"  dH^z)  =  X(l) 

-QC 

(3.  53) 

The  above  equations  are  the  essential  formal  difference  between  the 
nonregular  and  regular  ROC  characters.  For  the  nonregular  ROC 
the  value  of  one  or  both  of  these  integrals  will  be  less  than  one,  indi¬ 
cating  that  portion  of  the  decision  which  is  error  free.  TH  regular 
ROC  character  always  has  a  value  of  unity  for  the  complete  integral 
(Eqs.  3.8,  3.9). 

Two  special  ROC  curves  with  discrete  ROC  character 
have  been  of  importance  in  the  psychophysical  literature.  Both  are 
related  to  nonregular  ROC  characters  with  just  one  value  of  log  like¬ 
lihood  ratio  other  than  ±oc. 

The  pure- threshold  or  traditional- threshold  ROC  has 
only  two  values  of  log  likelihood  ratio  that  play  any  role,  +cc  and  some 
negative  value  Zq.  The  ROC  character  jump  is 

.  55 jq 

u„(Z0)  =  e  (3.54) 

Equations  3.  52  and  3.  53  are  applied  to  this  nonregular  ROC  charac¬ 
ter  to  determine  that  X(l)  is  one.  moaning  that  the  ROC  curve  first 
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contacts  the  upper  edge  at  the  corner  point  (1, 1).  The  ROC  curve  is 
given  by  Eq.  3.  55,  and  shown  in  Fig.  3.  6. 

Z0  Z0 

Y  =  1  -  e  u  +  e  u  X  (3.  55) 

The  more  common  form  of  this  equation  may  be  given  by  relabeling 
Y(0)  as  pQ>  sometimes  called  the  "true  probability  of  perception.  " 

Y  =  PQ  +  (1  -  PQ)  X  (3.  56) 

The  value  Zq  and  its  ROC  character  jump  are 

z0  “  In  (1  -  pQ),  co^Zq)  =  (1-P0)‘5  (3.57) 

A  second  nonregular  ROC  curve  related  to  a  single 
point  on  the  log  likelihood  ratio  axis  is  the  Green  ROC.  This  is  the 
nonregula  limit  of  D.  M.  Green's  double  threshold  ROC  curve.  For 
any  point  zy  and  any  ROC  character  jump  value  small  enough  that 


.  5  z 

o 

e 


(3.  58) 


The  formal  expressions  for  the  Green  ROC  curve  are 
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The  ROCcurve  is  sketched  in  Fig.  3.7  for  a  slightly  negative  value 

of  z  . 
o 

3.  6  Summary  of  Chapters  I,  II,  and  HI 

The  purpose  of  this  research  was  to  furnish  a  variety  of  ROC 
curves,  to  develop  sufficient  analytic  structure  to  classify  ROC  curves 
into  families  and  to  provide  a  means  of  generating  new  families.  In 
Chapter  II  it  was  shown  that  the  distribution  of  a  decision  variable 
may  be  quite  arbitrary  under  one  cause  condition;  therefore  decision 
models  may  be  developed  with  considerable  freedom. 

The  ROC  curve  contains  information  sufficient  to  specify  a 
decision  model  iff  the  decision  axis  bears  a  specific  functional  relation 
to  the  likelihood  ratio  f.  This  research  has  concentrated  on  z,  the 
logarithm  of  the  likelihood  ratio,  because  the  distributions  of  z  are 
necessarily  sufficiently  concentrated  to  possess  moment  generating 
functions.  It  has  been  demonstrated  that  the  ROC  character  can  be 
used  to  structure  ROC  curves  into  families,  and  to  provide  both  inter  - 
family  relationships  and  a  basis  for  generating  new  ROC  families. 

This  completes  the  development  of  the  research  into  the  structure 
of  ROC  curves.  The  following  chapters  deal  with  specific  ROC  families, 
in  order  to  determine  properties  peculiar  to  each,  and  to  demonstrate 
the  techniques  for  working  with  ROC  curves  and  ROC  families. 


CHAPTER  IV 


TRADITIONAL  ROC  FAMILIES 

This  chapter  will  deal  with  those  ROC  families  that 
appeared  in  Ref.  t. 

4. 1  Normal  ROC 

The  normal  ROC  has  been  used  extensively  in  both  the 
electronic  and  psychophysical  literature  (Refs.  1,  4,  7  through  14). 
This  type  of  ROC  curve  was  used  so  extensively  in  the  original  psycho¬ 
physical  work  of  Tanner,  S^ets,  and  Green,  that  many  thought  it  was  a 
necessary  part  of  their  perception  theory.  Tables  of  the  normal  ROC 
curve  are  available  (Refs.  17  and  18). 

The  ROC  character  for  the  normal  ROC  curve  is 


7t(z)  =  e 


i  i 

8  - - 

\  2v  d 


-  co  •  z  3C 


d  >  0 

(4.1) 

Multiply  the  character  by  e  -  ’  to  obtain  the  probability  density 
functions  for  the  logarithm  of  the  likelihood  ratio  under  the  two  conditions 
N  and  SN.  Both  of  these  are  normal. 
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(z  +  .  5d)2 

f(z  N)  =  --JL  e  (4.2) 

V  2zrd 

(z  -  .  5d)2 

-  ’  Sid 

f(ziSN)  =  ~~ —  e  (4.3) 

v  2nd 

with  equal  variance  and  with  mean  values  shifted  plus  and  minus  .  5d  from 
the  character  mode  of  zero.  The  ROC  curve  is  obtained  by  direct  inte¬ 
gration  of  Eqs.  4.  2  and  4.  3. 

P("A"  !N)  =  x  =  S  f— 5-d-~—  ) 

Vd 

P("A"  i  SN)  =  y  =  a(~^-)  (4.4) 

v  vt 

The  symbol  <E>  stands  for  the  normal  distribution  function.  From  this 
point  on,  the  probability  notation  for  the  ROC  curve  will  usually  be  omitted 
and  only  the  description  of  the  ROC  curve  as  a  real  function,  y,  of  a  real 
variable,  x,  retained.  While  Eq.  4.  4  explicitly  indicates  the  relation 
between  the  coordinate  values  and  the  logarithm  of  the  likelihood  ratio, 
this  relation  is  not  always  desired,  z  may  be  suppressed,  and  the 
equation  for  the  ROC  curve  written  as 

y  =  <J>(A  +  v^d" )  when  x  =  $(A)  (4.5) 


>vhere  the  variable  A  is  the  dummy  parameter  along  the  ROC  curve.  In 


102 


this  form  the  natural  parameterization  is  by  /d\  which  is  normally 
called  d'  in  the  psychophysical  literature. 

Graphs  of  the  normal  ROC  are  displayed  in  Figs.  4. 1 
to  4.  3.  In  4. 1  they  are  displayed  on  ordinary  linear  coordinate  paper, 
with  curves  given  for  d  values  of  zero  (the  chance  diagonal)  and 
d  =  1,  d  =  4,  and  d  =  9.  This  reflects  the  natural  stepping  shown  in 
Eq.  4.  5.  The  coordinate  paper  for  Fig.  4.  2  has  been  called  "normal- 
normal  paper"  indicating  that  it  is  related  to  the  normal  distribution 
function  on  both  axes,  and  is  also  called  "double- probability  paper" 
and  "z- scale  paper.  "  The  linear  distance  in  each  coordinate  direction 
on  this  paper  is  the  argument  of  the  normal  distribution  function.  Refer¬ 
ring  to  Eq.  4.  5,  on  such  paper  a  normal  ROC  point  plots  with  the 
vertical  coordinate  A  +  /cT,  and  horizontal  coordinate  X  .  These 
lie  along  a  straight  line  with  slope  one  and  separated  by  a  difference 
\  d~.  Transformation  to  normal  coordinates  spreads  the  unit  square 
over  the  entire  infinite  two-dimensional  plane.  The  region  displayed  in 
Fig.  4.  2  is  that  used  for  medium  probabilities,  between  one  percent  and 
ninety-nine  percent.  The  region  displayed  in  Fig.  4.3  is  more  common 

in  machine  application  where  very  small  probabilities  can  be  measured, 

-  6  “6 
and  extends  from  10  to  1  -  10  . 

The  main  diagonal  with  slope  plus  one  is  the  chance 
diagonal.  The  minor  diagonal,  with  slope  minus  one,  is  usually 
referred  to  as  the  "negative  diagonal"  and  corresponds  to  those  points 
where  the  probability  of  miss,  l  -  y,  is  equal  to  the  probability  of  false 


alarm,  x. 
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Fig.  4.3.  Normal  receiver  operating  characteristics 
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Four  normal  ROC  characters  are  plotted  in  Fig.  4.  4 
on  the  equivalent  of  semi-log  paper;  that  is,  In  7r(z)  is  plotted  against 
z  .  These  are  simple  parabolas  with  center  at  z  =  0  and  opening 
downward.  As  the  detectability  increases,  d  increases,  the  ROC 
characters  broaden  out,  giving  more  weight  to  large  magnitude  z 

_  0 

values.  The  locus  of  the  one  percent  false  alarm  probability  and  10 

false  alarm  probability  have  been  indicated.  Since  the  ROC  curves, 

and  ROC  characters,  are  symmetric,  the  negative  diagonal  corresponds 

to  z  =  0.  The  regions  of  interest  for  the  psychologist  will  normally  lie 

in  the  neighborhood  of  the  negative  diagonal  with  possible  extension  to 

high  positive  z  values  out  to  the  one  percent  point,  x  =  .01.  The 

region  of  interest  for  most  radar  and  sonar  applications  is  between  the 
-  6 

one  percent  and  10  false  alarm  probabilities.  (In  the  past,  some  have 

-14 

been  interested  in  false  alarm  probabilities  as  low  as  10  . )  These 

correspond  to  regions  of  high  positive  z  value. 

4.  2  ’’Case  II",  Detection  of  a  Sine  Wave  in  Added  Normal  Noise 

A  problem  occurring  frequently  in  the  electronic  literature 
is  the  detection  of  a  stable  sine  wave  with  uniformly  uncertain  phase  in 
the  presence  of  added  white  Gaussian  noise.  This  has  been  treated  by 
Rice.  (Ref.  19).  by  Marcum  (Ref.  16),  by  Middleton  (Ref.  7),  byHelstrom 
(Ref.  15).  was  included  in  Ref.  I  as  Case  II.  and  has  been  recently 
investigated  extensively  for  its  usefulness  in  hearing  by  LI>*yd  Jeffries 
(Ref.  20). 
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The  normal  ROC  case  has  been  used  so  easily  by  so 
many.  Yet  its  equation.  Eq.  4.5,  is  really  quite  complicated.  The 
normal  distribution  function  $  (  )  cannot  be  determined  in  terms  of 
polynomials  or  the  elementary  transcendental  functions.  It  owes 
whatever  simplicity  it  possesses  to  the  ease  with  which  tables  can  be 
obtained,  and  familiarity  with  this  specific  function.  In  the  case  at 
hand,  the  coordinates  of  the  ROC  curve  cannot  be  simply  written  in 
terms  of  each  other,  and  the  parametric  form  of  the  curve  again 
involves  a  tabulated  function.  However,  this  particular  function  i3 
one  with  which  few  people  have  great  familiarity  and  for  which  the 
tables  are  not  as  readily  available  as  one  would  wish.  Tables  have 
been  calculated  by  Marcum  (Ref.  16),  specifically  for  the  evaluation 
of  this  ROC  curve.  These  are  called  Q  Tables  since  that  is  the 
symbol  that  he  uses. 

Q  Tables:  x  =  e-f3  1  2  =  Q(0,  ft  )  -  f  t  e_t  '  2  dt 

y  =  Q  (a,  13)  =  /te'r  /2  e'a  2/2  IQ(at)  dt 

(4.  6) 

The  false  alarm  probability  is  a  very  simple  function  of  the  parameter 
of  the  curve,  /l  ,  but  the  detection  probability  involves  integration  of  a 
function  whose  integrand  contains  the  modified  Bessel  function  of  order 
zero,  itself  a  non-simple  tabulated  function.  The  parameter  a  ,  a 

t 

A 


109 


positive  real  number,  indicates  the  quality  of  detection.  As  a  in¬ 
creased,  the  ROC  curves  become  better.  The  Q  tables  were  used  to 
plot  the  ROC  family  of  Fig.  4.  5. 


Vd  Values 

Slope 

Formulae 

a 

At  Y  =  50;  q 

At  X  -  1  -  Y 

At  X  =  .  50 

s 

a-  . 

5  Vln  I0(Qf) 

0 

0 

0 

0 

1 

0 

.  5 

- 

- 

0. 108 

0.  970 

0 

- 

1.0 

0.412 

0.40 

0.  381 

0.  925 

0.  5 

0.486 

1.  5 

0.869 

- 

0.761 

0.877 

1.0 

- 

2.0 

1.397 

1.  30 

1.  188 

0.  851 

1.5 

1.  56 

2.  5 

1.940 

- 

1.640 

0.845 

2.0 

- 

3.0 

2.476 

2.30 

2.  106 

0.852 

2.  5 

2.65 

3.  5 

3.003 

- 

2.  581 

0.  861 

3.0 

- 

4.0 

3.  512 

3.35 

3.  544 

0.  872 

3.  5 

3.70 

4.  5 

4.001 

- 

4.036 

0.885 

4.0 

- 

5.0 

- 

4.40 

- 

« 

4.  5 

4.75 

5.  5 

— 

Graph 

Table 

— 

5.0 

■** 

6.  0 

- 

5.45 

- 

- 

5.  5 

5.78 

7.0 

- 

6.  47 

- 

- 

6.  5 

6.80 

8.0 

- 

7.  51 

- 

- 

7.5 

7.82 

9.0 

8.  55 

— 

■  ■  ,, 

8.  5 

8.84 

Note:  q  and  s  are  binormal  approximation  numbers.  See  Chapter  VII. 
Table  4.  1  Numbers  relating  Q- Table  ROC  to  normal 


The  v  d  and  slope  readings  for  a  range  of  a  are 
given  m  Table  4.  1  and  plotted  in  Fig.  4.  6. 

Consider  a  random  variable  t  which  ranges  from  zero  to 
infinity  with  probability  density  functions  given  by  the  integrand  in  Eq. 

4.  6.  it  is  immediately  evident  that  the  likelihood  ration  of  t  is  the 
product  of  the  third  and  fourth  terms  in  the  integrand  of  the  equation 
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for  y  (Eq,  4.  6).  Therefore,  the  logarithm  for  the  likelihood  ratio  of 
t  is 


z  =  In  Iq  (at)  -  .5a  2  (4.  7) 

and  is  strictly  monotone  increasing  with  the  variable  t.  Those 
familiar  with  the  chi-square  distribution  will  recognize  that  under  the 
N  condition  t  is  the  square  root  of  chi-square  with  two  degrees  of 
freedom  while  under  the  SN  condition  t  is  the  square  root  of  a  noncentral 
c  hi- square  with  two  degrees  of  freedom. 


t  =  VX2 


2d.  f. 


vs. 


t  =  VX' 


2d  f. 


(4.8) 


The  general  problem  of  detecting  a  shift  from  central  to  noncentral 
cni-square  distribution  with  a  known  number  of  degrees  of  freedom  has 
been  treated  in  Ref.  21.  The  variable  t  rather  than  chi-square  has 
been  chosen  for  two  reasons;  the  first  is  that  Marcum's  tables  use  t 
instead  of  t2  ,  the  second  is  that  the  Bessel  function  In  IQ(at)  is  nearly 
linear  with  its  argument,  at,  when  the  argument  is  large.  This  means 
that  the  density  function  for  z  should  be  of  the  same  type  as  the  density 
function  for  t  if  large  values  of  at  are  the  relevant  values. 

The  ROC  character  for  z  can  be  obtained  from  the  root 
likelihood  product  of  t  multiplied  by  the  Jacobian  of  transformation 
from  t  to  z.  The  root  likelihood  product  of  t  is  simply  the  N  density 
function  multiplied  by  e* 
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1  -  k- 


=  f(tlN)  e 


.  5z  i  dt 


(4.9) 


It  would  be  desirable  to  write  t  in  terms  of  z  and  determine  the  form 


of  the  ROC  character  as  a  function  of  its  argument,  z.  We  cannot  invert 


Eq.  4.  7  in  any  practical  fashion.  Therefore,  the  best  procedure  is  to 


write  the  ROC  character  in  terms  of  the  parameter  t,  and  to  attempt 


to  graph  or  approximate  the  ROC  character. 


,  v  .  -t 72  -a  / 

7f(z)  =  te  e 


2  /  2  T  .5  ,  ^  L0 


In  (at) 


V  <«*>  ? 


(4.10) 


Table  9.  8  of  the  NBS  Handbook  (Ref.  6)  gives  not  the 


modified  Bessel  function,  but  the  more  slowly  varying  quantity. 


T  (A)  =  e"A  I  (A) 

V  V 


(4.11) 


(T  has  been  used  to  indicate  "tabulated  function.  ")  In  order  to  use  the 


tables,  Eq.  4.10  is  rewritten  in  terms  of  the  table  entries.  The 


logarithm  of  the  Bessel  functions  can  be  written  as 


In  I  (A  )  =  In  T  (A )  +  A 

j.»  1/ 


(4.12) 


and  inserted  in  Eq.  4.  7  and  ' 


z  =  [  .n  Tgvui,  +  at]  j  -  [  .  5a2  ] 


(4.13) 
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In  7T  (z)  =  [  .  5(ot)  +  ln(cct)  +  1.  5  In  Tp  (at)  -  In  Tj(at)  j  2  ~  •  5t 


-[  .25  a  2  -  2  In  a  ]  (4.14) 


The  argument  of  the  ssel  function  is  at,  and  therefore  the  equations 


have  been  written  in  t'  xms  of  a  quantity  which  is  a  function  of  at  alone. 


and  then  modifiers  depending  upon  t  ora.  Since  a  is  the  parameter; 


of  an  ROC  curve,  it  will  be  constant  over  any  computations  of  the  ROC 


or  its  character.  These  equations  were  utilized  to  calculate  ROC 


characters  with  table  values  and  slide  rule.  The  resultant  ROC 


character  for  three  values  of  the  parameter  is  shown  in  Fig.  4.7.  This 


is  sho  vn  on  the  same  coordinate  system  that  was  used  for  the  normal 


ROC  character  Fig.  4.  4.  The  range  of  the  random  variable  z  is 


bounded  below  by  .  5a  2  as  can  be  seen  in  Eq.  4. 13.  and  hence  the  ROC 


curve  will  have  a  nonzero  minimum  slope. 


Since  the  ROC  character  is  bounded  below  but  unbounded 


above,  it  cannot  possibly  be  symmetric.  The  mode  of  the  ROC  character 


for  the  cases  shown  is  slightly  negative.  In  the  case  a  =  1  the  mode 


is  at  the  extreme  negative  value,  z  =  -.  5.  For  the  cases  a  =  2  and 


a  -  \  10  ,  the  mode  values  are  approximately  z  =  -  .13,  and  z  =  -.  79, 


respectively.  As  detectability  increases  ( a  increases)  the  logarithm  of 


the  ROC  character  covers  a  wider  and  wider  range  and  begins  to  appear 


similar  to  those  for  the  normal  case.  It  is  therefore  very  tempting  to 


attempt  to  fit  the  ROC  curve  with  a  normal  ROC  curve,  to  fit  the  ROC 


character  with  a  normal  ROC  character.  In  Ref.  22,  the  approximation 


A--' 


*»%*«*• 
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d  =  {a  -  .  5)2  has  been  given.  Some  have  used  the  approximation 

d  -  In  Iq  (a  2 ).  Both  of  these  have  been  useful  in  engineering  application 

to  indicate  that  performance  is  very  poor  for  a  less  than  .  5,  and 

essentially  normal  for  a  greater  than  /10.  For  those  interested  in 

-  6  -  2 

the  ROC  curve  between  false  alarm  probabilities  of  10  and  10  the 
picture  is  simpler,  since  it  is  evident  from  Fig.  4.  7  that  over  this 
region  the  arcs  of  the  logarithm  of  the  ROC  character  could  fit  quite 
well  with  a  parabola.  Appendix  C  contains  the  numerical  work  in  an 
attempt  to  fit  the  entire  a  -  2,  In  n(z)  curve  over  the  portion  shown 
in  Fig.  4.  7.  This  means  that  the  same  expression  must  fit  near  the 
mode  as  well  as  aL  ng  the  skirt.  The  result  is  summarized  in 
Eq.  4.15. 


n(z)  ~  .  25  e 


z  +  .  725 
1.625  " 


1.885 


(4.15) 


1  he  numuers  .  725  and  .  25  come  directly  from  the  graphically  determined 
mode.  The  power  l.  885  is  close  to  2.  000.  This  accounts  for  the  normal 
appearance  of  the  ROC  curve  and  ROC  character  over  short  segments. 

The  difference  between  2.000  and  1.  885  accounts  for  the  failure  of 
attempts  to  match  the  ROC  curve  or  its  character  over  major  portions. 

4.3  Noise-in-Noise,  Same  Spectrum 

There  are  two  basically  different  problems  called  noise- 
in-noise  in  the  detection  literature.  The  first,  and  the  one  that  we  shall 
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consider  here,  is  the  case  which  is  an  increase  in  the  power  level 
without  any  other  change  in  the  statistics  of  the  observation.  The 
contrasting  case,  not  considered  here,  is  when  the  two  noise  pro¬ 
cesses  differ  in  spectrum  and  in  autocorrelation  function.  This 
latter  case  has  caused  considerable  controversy  in  the  literature 
because  one  can  obtain  singular  detection  at  lew  power  levels  from 
seemingly  innocent  assumptions. 

The  standard  problem  assumes  2WT  independent 
samples  of  a  Gaussian  noise  process  are  observed.  Each  sample 
has  the  probability  density  function  given  by  Eq.  4. 16. 

u.2 

l 

f(u.;N)  = e  2N  (4.16) 

1  v  2ttN 

(I  apologize  for  the  confusing  notation  which  uses  N  to  indicate  the 
cause  on  the  left-hand  side  of  the  equation,  and  to  represent  the  noise 
power  or  variance  on  the  right-hand  side  of  the  equation;  however,  this 
is  the  standard  notation  in  the  field. )  The  signal  to  be  detected,  if  it 
were  observed  noise- free,  would  be  of  the  same  nature 


f(u  S)  =  -  rrLzr  e  ^  (4.17) 

1  \  2v  S 

except  that  it  has  a  power  S.  When  the  signal  is  added  to  the  noise,  the 
resudant  observations  will  have  the  probability  density  function  given 


118 


by  Eq.  4. 18. 


f (u .  ,  SN )  =  •  - — -  e2(N  +  S)  (4.18) 

1  V2ff(N+  S) 

The  logarithm  of  the  likelihood  ratio  of  an  individual  point  observation 
is  obtained  by  dividing  and  taking  logarithms. 

2 

z(ui>  =  ~2~  'W  "  N  +  S  ^  +  2  ln  ^r+!>)  (4-19) 

If  a  number  of  independent  observations  are  to  be  used 
to  reach  a  single  decision,  then  the  likelihood  ratio  of  the  total  obser¬ 
vation  is  the  product  of  the  likelihood  ratios  of  the  individual  parts. 

This  means  that  the  logarithm  of  the  likelihood  ratio  of  the  total 
observation  will  be  the  sum  of  the  logarithm  of  the  likelihood  ratios  of 
the  individual  observations.  It  follows  directly  from  Eq.  4  19  that  the 
sum  of  the  z  values  corresponds  to  the  sum  of  the  squares  of  the 
individual  observations. 


2WT 

u  =  £  u2  (4.20) 

i=l  1 


Thus,  u  is  a  sufficient  statistic.  It.  follows  directly  that  the  log  of 
the  likelihood  ratio  of  u  is  the  sum  of  the  logarithms  of  the  likelihood 


ratios  of  the  individual  observations. 
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Since  u  is  the  sum  of  the  square  of  Gaussian  random  variables,  its 
distribution  is  closely  related  to  the  chi-square  distribution  with  2WT 
degrees  of  freedom.  Specifically,  the  variable  u/N  will  have  a  chi- 
square  distribution  with  2WT  degrees  of  freedom.  This  can  be  used 
to  obtain  the  distribution  for  u  . 


In  a  similar  manner,  the  distribution  for  u  under  condition  SN  is 
obtained  from  u,  (S  +  N)  being  a  chi-square  random  variable.  The 
equation  is  exactly  like  Eq.  4.22,  except  that  N  is  replaced  by  (N  +  S). 

The  present  objective  is  not  to  obtain  the  individual 
density  functions  to  take  their  ratio,  since  we  already  know  the  likeli¬ 
hood  ratio.  The  objective  is  to  obtain  the  individual  density  functions 
in  order  to  obtain  their  product.  Since,  by  Eq.  4.  21,  u  is  a  linear 
translation  of  Z,  the  ROC  character  is  obtained  from  the  root 
likelihood  product  for  u  by  multiplying  it  by  the  Jacobian  of  trans¬ 
formation  (the  derivative  of  u  with  respect  to  Z. ) 


n(z)  =  [f(u  N)f(u  SN)  j  ^ 


(4.23) 


r 


► 


120 


From  Eq  4.  21,  z  is  bounded  below  by  a  quantity  we  shall  call 
because  u  is  bounded  below  by  zero. 


Z0  =  WTln(NT-S)  =  -WTln(l+|) 


(4.24) 


From  (4.  22)  and  (4.  23) 


v(z)  = 


N-5WT(S  +  N)-5WT 


r(WT)  s 


WT 


u  S 
2N(N  + 


1  WT- 1  - 


sy 


u  S  2N  +  S 
2N(N  t  S)  2S 


(4.25) 


\ 


The  next  step  simply  is  the  change  of  variables  from  u  to  z. 


nlz)  = 


1 

r(WT) 


N(S  +  N) 


T  .  5WT 


(z 


z0)WT-X  e 


-( 


N 


5)(  z  -  ZQ) 


(4.26) 


This  ROC  character  is  a  Pearson  III  character,  a  two- 
parameter  class.  When  WT  is  small,  or  S/N  is  large,  the  ROC 
curve  can  be  obtained  by  direct  integration.  Tables  of  the  chi-square 
distribution  may  be  used  to  plot  the  ROC  curves. 


X 


1  -  P(x2 


2WT) 


j 
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When  S.  N  is  very  small,  but  WT  is  sufficiently  large 
so  that  the  resulting  performance  corresponds  to  normal  indices  of 
the  order  of  0.  5  to  50,  a  normal  approximation  to  the  ROC  curves  may 
be  made.  The  formula  for  the  matching  normal  index 

3  =  (WT  -  1)  J  (4.28) 

is  derived  in  Appendix  D,  using  the  ROC  character.  This  approximation 
is  not  new;  the  use  of  the  ROC  character  to  obtain  it  is  new. 


4.  4  Signal  One  of  M  Orthogonal  Signals 

Chapter  IV  on  ROC  models  derived  from  electronics 
cases  concludes  with  a  review  of  the  M  orthogonal  signals  case.  The 
situation  is  this.  An  observation 


1 

I 


X  =  (Xr  (4.29) 

consists  of  M  similar  parts.  If  ihe  condition  is  noise  alone,  N,  the 
individual  parts  are  similarly  distributed  and  are  statistically  independ¬ 
ent 


M 

f  (X  N)  =  II,  f,(X,  N)  (4.30) 

‘Vi  A.  a.  1 

Therefore,  the  probability  density  for  the  occurrence  of  the  total 
ob.M  r\  at  ion  is  given  by  a  product  of  the  individual  N  probability 


A 
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M  different  subcauses,  possibly  M  specific  signals.  Each  possible 
subcause  affects  only  one  part  of  the  observation.  Thus,  if  the  j-th 
signal  is  present,  the  probability  density  for  the  total  observation  is 
given  by 

f  (X.S  N)  =  MX  ’SN)  II  fj  (X.  N)  (4.31) 

The  j-th  signal  affects  only  the  j-th  part  of  the  observation  and  the  rest 
of  the  observation  is  distributed  the  same  way  it  was  under  the  condition 
N  Rewrite  Eq.  4.  31  in  terms  of  Eq.  4.  30  by  multiplying  and  dividing 
by  the  missing  factor  in  the  large  product  and  obtain 

fM(XiS  N)  =  ^(X.Jf^XiN)  (4.32) 

v 

for  the  probability  density  under  the  special  subcondition  of  the  j-th  SN 
cause.  The  actual  SN  cause  is  an  ensemble  of  these  subcauses,  each 
occurring  with  known  probability  P(S^N).  The  probability  density 
function  for  the  total  observation  under  the  condition  SN  is  therefore 
the  average  probability  density  function  (averaged  over  the  various  sub¬ 
causes.  ) 
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Since  the  term  f  (XIN)  is  independent  of  the  summation,  divide 
through  by  it  to  obtain  the  likelihood  ratio  for  the  total  observation. 

*(X)  =  £  P(S.N)  ft(X  )  (4.35) 

j  J  J 

The  logarithm  of  the  likelihood  ratio  is  simply  the  logarithm  of 
Ea-  4.35 

z. 

z  -  In  V  p  (S  N)  e  j  (4.36) 

j  3 


The  physical  picture  of  the  receiver  for  such  a  situation 
is  one  of  M  parallel  branches,  each  of  which  computes  a  log  likelihood 
ratio  on  part  of  the  observation.  Each  of  the  log  likelihood  ratios  is 
passed  through  an  exponential  nonlinearity  which  greatly  emphasizes 
large  positive  values.  The  output  of  these  nonlinearities  are  averaged 
together  to  form  the  total  likelihood  ratio  of  the  observation.  If  this 
total  likelihood  ratio  is  passed  through  a  logarithmic  nonlinearity,  the 
output  will  again  be  on  a  z  axis.  Although  the  value  of  z  on  any 


observation  will,  in  general,  be  less  than  the  maximum  individual  log 
likelihood  ratio,  the  effect  of  the  peaking  function  before  the  averaging 
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The  evaluation  of  reasonable  parameter  cases  for  the 
M  orthogonal  signal  problem  s  taxed  many  people.  The  difficulty 
is  the  presence  of  the  nonlinear  it  ies  before  the  averaging  process. 

Some  authors  have  felt  that  the  presence  of  the  averaging  process 
would  tend  to  make  the  output  normally  distributed.  However,  if  one 
realizes  that  £  ranges  over  positive  values,  has  an  expected  value  of 
one  for  condition  N,  a  good  normal  fit  could  be  obtained  only  if  the 
standard  deviation  of  the  fit  is  small  compared  to  one.  The  direct 
route  to  obtain  the  distribution  of  z,  or  £  ,  is  to  carry  through  the 
M-fold  convolution  to  obtain  the  results.  This  could  be  done  in  the 
transform  plane  by  finding  the  characteristic  function  for  £,  raising 
to  a  power,  and  transforming  back.  Since  the  likelihood  ratio  often 
fails  to  possess  a  moment  generating  function  one  suspects  that  a 
great  deal  of  care  will  ee  necessary  if  any  approximation  or  numerical 
work  is  done  in  obtaining  the  characteristic  function  or  its  inverse. 

An  Example :  Let  us  consider  a  particular  example  of 
ROC'  character  for  the  individual  parts  of  the  observations  that 
simplify  me  process  of  obtaining  the  ROC  character  for  the  total 
observation.  Consider  the  ROC  equation  given  in  Eq.  4.38 

yl  =  X1  ’  X1  ln  xi  ^ 

and  graphed  on  normal-normal  paper  in  Fig.  4.  8  This  does  not 
represent  particularly  good  observation  quality,  or  any  physical  case 
of  interest,  but  has  been  chosen  because  of  the  specific  form  of  its 


i 


Fig.  4.8.  A  particular  ROC  curve 
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ROC  character. 


1. 5  z.  i 

77 1 ( z  ^)  =  e  1  e  e  -oo  <  z.  <  oc  (4.39) 

—  5z 

Multiply  the  character  by  e  '  to  obtain  the  probability  density  function 
for  z  under  the  condition  N, 


z. 

L  (z.  i  N)  =  e  1  e  e 

1  l 


z. 

i 


(4.  40) 


and  by  substitution,  obtain  the  probability  density  function  for  the 
individual  part-of-the- observation's  likelihood  ratio. 

-Si.  .  -j {. 

fjUj  N)  =  Si  e  1  £  =  e  1  0  <  £.  •  oo 


(4.  41) 


Let 


u 


(4.  42) 


The  distribution  of  u  is  obtained  by  simple  use  of  the  characteristic 
function.  The  resulting  density  function  is 

M  -  1  -u 

«U|N>  -  ~  (M  -  lj! '  (4'43) 

The  likelihood  ratio  desired  is  not  the  sum  of  the  individual  likelihood 
ratios,  but  the  average  likelihood  ratio.  Assume  that  the  subcauses  are 
equally  likely.  From  Eq.  4.  43  obtain 
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M 

U  =  Mf,  f(£  !  N)  =  (M " -f)';"  *  e  (4.44) 

From  the  distribution  for  the  likelihood  ratio,  transform  to  the  log 
likelihood  ratio  axis 

w„ixn  M  JMz  -Me  , . 

f(Z;N)  =  ^M"-T)T  6  6  (4’45) 

5z 

and  multiply  by  e'  to  obtain  the  ROC  character  for  the  total  observa¬ 
tion  procedure. 


M 


M 


11 (M  -  1)! 


(M  +  .  5)z  e“Me 


-oo  <f  z  <  oo  (4.  46) 


This  ROC  character  is  functionally  similar  to  the  single  observation 
character. 

1.  5  z.  z. 

i  —  0  t 

77^(z.)  =  e  e  -oo  <  z  <  oo  (4.47) 


Normal  Case:  To  obtain  an  accurate  estimate  of  the  ROC 
curve  when  the  individual  observations  are  normally  distributed, 
Jaarsma  (Ref.  24)  programmed  the  IBM  7090  to  obtain  the  distributions 
using  the  characteristic  function.  Reproduced  here  are  some  of  his  ROC 
curves  and  the  ROC  characters  for  the  particular  case  when  the  original 
observation  was  normally  distributed  with  parameter  d  =  4.  The  ROC 
character  for  this  individual  observation  is  given  by  Eq.  4.  48. 
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"  K  i  ~  -T75 — 

tt1  (z.)  =  e  — —  e  -oc  '  Z  <  00  (4.  48) 

1  1  v  8tt 

ROC  curves  are  for  5,  20  and  100  parallel  channels  of  observation.  The 
resulting  performance  is  displayed  on  normal-normal  paper  in  Fig.  4.  9. 
The  ROC  characters  for  these  three  cases  are  drawn  on  linear  paper  in 
Fig.  4. 10.  The  three  characters  all  have  the  same  general  form.  They 
are  unimodal  with  the  mode  being  at  a  slightly  negative  value  of  Z.  The 
decrease  from  the  mode  is  faster  in  the  negative  direction  than  in  the 
positive  direction.  As  the  number  of  parallel  channels,  M,  increases 
and  the  detection  performance  falls  off  the  ROC  character  is  essentially 
confined  to  a  narrower  range  of  z  values  and  becomes  correspondingly 
more  peaked.  To  compare  these  ROC  characters  with  those  shown  for 
Cases  I  and  n  the  logarithm  of  the  ROC  character  has  been  plotted  in 
Fig.  4. 11  to  the  same  scale  as  for  Fig.  4.  4  and  Fig.  4.  7.  Because  of 
the  specific  mechanics  of  the  computer  program,  the  computation  of  the 
character  was  terminated  at  the  neighborhood  of  the  one  percent  false 
alarm  values;  that  is,  x  =  ,  01.  In  order  to  look  at  one  of  these  in  more 
derail,  the  axes  were  changed  and  the  ROC  character  replotted  for  the 
case  of  M  =  20  in  Fig.  4. 12.  In  Fig.  4. 12,  the  z-range  shown  extends 
only  from  -2  to  +2.  To  the  left  of  the  mode,  the  appearance  of  the  ROC 
character  is  quite  parabolic  (or  at  least  power  law).  The  fall-off  to  the 
positive  z  direction  is  shallower,  as  has  already  been  noted,  and 


\ 


ROC  c 


appears  to  be  more  linear. 


In  order  to  investigate  this  nearly  lineai  fall-off  of  the 
log  of  the  ROC  character,  consider  what  the  effect  would  be  if  the 
ROC  character  behaves  exponentially  for  large  positive  values. 

Assume 

-C  z 

7T (z)  =  Cx  e  Cx  >  0,  C2  >  .  5,  z  >  C3  (4.  49) 


holds  for  z  greater  than  some  value,  say  C^,  and  with  coefficients 
C^  and  Cg.  For  this  simple  ROC  character,  the  ROC  curve  can  be 
obtained  by  direct  integration 


00 


5t 


7T  (t)  dt  = 


-(C2t  .5)  z 


e2T  .5 


(4.  50) 


Taking  logarithms  of  both  sides 

In  (£)  =  In  Cj  -  In  (C2  +  .  5)  -  (C2  t  .  5)  z  (4.  51) 


Eliminate  the  parameter  z  to  obtain  a  nonparametric  form  for  the  ROC 
curve. 


In  X 


+  .  5 
"75 


)  In  Y  -  ( 


5>  ln  C1 


+ 


5 

-g  )  -  In  (C2  +  ■  5) 


(4.52) 


This  shows  that  the  ROC  curve  would  plo',:  as  a  straight  line  on  log- log 
paper  for  that  range  for  which  the  ROC  character  is  essentially  a 
simple  exponential.  The  general  form  for  such  a  curve  is  given  by 
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In  X  A  In  Y  +  In  B  =$  X  =  BYA  (4.  53) 

The  ROC  curves  of  Fig.  4.  9  are  replotted  on  log- log 
paper  in  Fig.  4.  13.  'I  he  straight-line  portions  of  the  ROC  continue  well 
beyond  the  initial  part  of  the  curve  up  to  the  general  region  of  Y  =  .  55. 
There  must  be  a  departure  from  this  straight  line  form  as  the  ROC 
curve  approaches  the  point  (1,  1)  since  the  ROC  curve  is  regular,  and 
the  ROC  character  is  definitely  not  exponential  over  its  complete  extent. 
Figure  4. 13  does  indicate  that  the  exponential  portion  of  the  ROC 
character  extends  amazingly  close  to  the  mode. 

A  detection  situation  where  the  likelihood  ratio  is  an 
average  likelihood  ratio,  corresponding  to  the  signal  affecting  only  one 
of  many  parts  of  the  observation,  leads  to  great  difficulty  in  treating 
the  ROC  analytically.  This  is  in  contrast  to  the  happier  situation  when 
the  logarithm  of  the  likelihood  ratio  is  the  sum  of  the  logarithm  of  the 
likelihood  ratio  for  various  parts  of  the  observation,  corresponding  to 
the  situation  where  the  signal  affects  all  of  the  individual  parts  of  the 
observation.  In  this  latter  case,  one  will  be  dealing  with  the  distribution 
for  z,  a  variable  which  has  concentrated  density  function,  moment 
generating  functions,  and  tends  to  be  more  amenable  to  transform 
methods  than  £ ,  which  has  wildly  skewed  probability  distributions. 


•i  y  W  \  I 


-**|N*-  yfm 


CHAPTER  V 


ALGEBRAIC  ROC 

This  chapter  is  dedicated  to  Professor  James  Egan.  His 
work  with  the  type  of  ROC  known  as  the  Power  ROC  (Ref.  23)  suggested 
the  use  of  simple  algebraic  formulas  for  use  in  ROC  models.  Such 
models  can  be  expected  to  be  used  to  fit  data,  or  approximate  the  be¬ 
havior  of  a  decision  mechanism,  and  not  necessarily  arise  from  phys¬ 
ical  phenomena.  Egan  noted  the  nonnormal  nature  of  ROC  curves  that 
he  obtained  for  human  observers.  The  region  of  the  ROC  near  the  (0,0) 
point  appeared  to  be  fairly  normal,  sharply  rising  and  curved.  How¬ 
ever.  the  region  of  the  ROC  near  the  upper  corner  (1,  1)  was  more  like 
the  pure  threshold  ROC  curve  of  traditional  psychophysics,  with  the 
curve  approaching  the  upper  corner  at  some  nonzero  slope.  ROC 
curves  of  this  nature  are  shown  in  Fig.  5.  1.  These  curves  are  Egan 's 
pure  power  ROC  curves,  detailed  in  Section  5.  1. 

The  description  of  the  ROC  curve  as  an  algebraic  for¬ 
mula  will  mean  that  most  of  the  mathematics  will  be  simpler  than  that 
necessary  when  the  ROC  is  described  by  higher  transcendental  equa¬ 
tions.  The  calculus  that  is  involved  is  generally  elementary.  The 
algebraic  ROC  curves  are  recommended  as  classroom  examples 
since  little  higher  mathematics  is  required  of  the  student. 

In  this  chapter  Egan’s  pure  power  ROC  i  examined  in 
Section  5.  1.  obtaining  a  one  parameter  family  of  ROC  curves.  In  5.2 
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Extrr.  Curves 


Locus  of  f  =  1.  z  -  0 
Negative  diag  X  =  1  -  Y 


g.  5. 1.  Power  ROC  curves  on  linear  graph  paper; 
X  =  YA 


the  general  conic  ROC  curve  is  considered.  These  ROC  curves  are 
arcs  o<  parabolas,  circles,  ellipses,  or  hyperbolas.  The  Luce  ROC 
Curve  will  be  considered  a  conic  section,  it  being  the  two  straight  line 
asymptotes  of  a  hyperbola.  The  regular  conic  ROC  is  a  three  parame¬ 
ter  family.  Section  5.3  is  devoted  to  a  nonlinear  scaling  which  can  be 
used  tc  produce  a  special  type  of  probability  paper,  called  lor- lor 
graph  paper.  The  Chapter  concludes  with  Section  5. 4  on  curve¬ 
fitting  using  the  conic  ROC  curves  While  the  three  parameters  of 
the  conic  ROC  family  are  too  cumbersome  to  yield  a  simple  set  of 
ROC.  they  become  three  degrees  of  freedom  in  fitting  curves  to  data. 

5.1  Power  ROC  :  X  = 

The  ROC  curve  of  the  pure  power  type,  X  =  Y^,  where 
A  is  greater  than  one,  has  already  been  shown  in  Fig.  5.  1  on  linear 
graph  paper.  In  this  graphical  form  it  is  similar  to  many  ROC  curves 
that  have  been  measured.  This  type  of  ROC  has  one  property  (Ref.  23) 
which  is  extremely  convenient  in  situations  where  the  probabilities. 

X  or  Y.  can  be  determined  except  for  a  fixed,  unknown,  constant  of 
proportionality. 

For  example,  in  an  experimental  situation  in  which  the 
number  oi  false  alarms  is  measured,  but  the  number  of  opportunities 
for  false  alarm  can  only  be  estimated,  the  false  alarm  probability,  X, 
can  not  be  determined  exactly.  If  tvo  similar  experimental  runs  are 
made,  or  if  an  experimental  technique  is  used  so  that  two  points  on 
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the  ROC  can  be  determined  simultaneously,  then  one  can  obtain  the 
value  for  A  without  determining  the  exact  number  of  opportunities  for 
false  alarm.  If  each  of  these  two  operating  points  fall  on  the  power 
ROC .  then  so  does  th«  r  ratio 

Once  it  is  known  that  the  ROC  curve  is  a  power  law  ROC  curve,  and 
the  parameter  A  determined  from  Eq.  5.  1  using  two  operating  points, 
then  the  number  of  opportunities  for  false  alarm  can  be  obtained  from 
either  one  of  the  original  ROC  data  points. 

ROC  Curves.  Equation  5.  1  suggests  the  use  of  log-log  paper,  since 
the  equation  will  plot  as  a  straight  line  on  such  paper.  Power  ROr 
curves  with  A  parameter  values  of  1.  2,  3,  4.  8,  16,  32,  64,  are 
shown  in  Fig.  5.  2,  on  log- log  paper.  In  addition  to  the  ROC  curves, 
the  locus  of  the  proper  operating  point  for  a  symmetric  bet,  z  =  0, 
and  the  position  of  the  negative  diagonal  points  are  shown.  Log- log 
paper  may  be  useful  when  A  is  less  than  eight  or  ten,  but  the  crowd¬ 
ing  on  the  graph  will  negate  its  usefulness  for  higher  values  of  A.  If 
extremely  low  false  alarm  rates  are  being  investigated,  then  larger 
A  values  will  be  feasible.  Probability  of  detection  values  less  than 
.  80  can  be  plotted  well  on  this  paper. 

Pure  power  ROC  curves  have  been  drawn  on  Fig.  5. 3 
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on  normal-normal  paper.  These  curves  are  "amazingly  straight." 
The  crowding  as  the  value  of  A  is  doubled  is  not  as  pronounced  as 
on  log- log  paper.  If  one  had  experimental  data  lying  along  one  of 
these  pure  power  ROC  curves  plotted  on  normal- normal  paper  in  the 
range  shown,  one  might  well  come  to  the  conclusion  that  the  "true" 
ROC  curve  was  a  straight  line  on  normal- normal  paper,  but  with 
slope  less  than  one.  The  strict  interpretation  of  such  truly  straight 
lines  on  normal- normal  paper  will  be  considered  in  Chapter  VI  on 
binormal  ROC  curves.  The  negative  diagonal  on  linear  paper  is  also 
the  negative  diagonal  on  normal-normal  paper.  The  locus  of  the  unity 
slope  values,  where  z  =  0.  plot  along  a  line  almost  straight  and  par¬ 
allel  to  the  negative  diagonal.  If  a  human  observer  had  a  pure  power 
ROC  curve,  and  operated  optimumly  in  a  fair  bet  yes- no  detection 
experiment,  he  would  operate  somewhere  along  this  z  =  0  line  and 
the  experimenter  would  conclude  that  the  observer  "had  a  bias  toward 
the  'B'  decision.  " 

A  common  procedure  in  both  psychophysical  work  and 
in  binary  communication  work  is  to  relate  performance  along  the  neg¬ 
ative  diagonal  to  either  the  index  of  the  curve,  to  the  signal  to  noise 
ratio  of  the  presentation,  or  to  some  other  indicator  of  goodness  of 
performance.  Along  the  negative  diagonal  the  probability  of  a  miss, 

1  -  Y.  is  the  same  as  the  probability  of  a  false  alarm.  X.  For  the 
pure  power  ROC  curve  with  index  A  the  probability  of  detection  along 
the  negative  diagonal  will  satisfy  Eq.  5.  2. 


143 


* 


1  -  Y  =  YA  (5.2) 

Equation  5.  2  relates  the  index  A  to  the  ROC  coordinates  on  the  nega-  \ 

live  diagonal.  This  was  solved  numerically  and  the  results  plotted 
in  Fig.  5.  4. 

Another  common  practice  is  to  convert  the  negative 

diagonal  point  to  a  normal  index  of  detectability.  This  index  is  labeled 

dp  because  it  is  related  to  the  point  of  equal  error.  Figure  5.  5  is  a 

plot  of  the  power  ROC  index  A  vs.  \/d  .  Although  there  is  some  cur-  ^ 

vature  in  this  graph,  a  first  approximation  is  that  the  logarithm  of  A 

is  roughly  proportional  to  the  square  root  of  dg.  For  low  index  of 

detection,  one  might  use  the  approximation  that  Vdg  =  2  log^  A. 

Although  such  practices  of  reporting  a  normal  index  of  detection  for 

nonnormal  ROC  curves  may  be  commendable  because  it  increases 

general  communication  among  experimenters  using  different  models. 

it  carries  with  it  the  risk  that  some  uninformed  people  will  assume 

that  the  ROC  curve  is  truly  normal.  The  index  dg  should  always  be 

accompanied  by  the  statement  that  dp  applies  only  to  the  single  point 

on  the  negative  diagonal.  This  is  the  practice  of  most  user's  of  the 

index  d  . 
e 

ROC  Character.  Two  basic  properties  of  any  ROC  curve  are  (1)  the 
ROC  curve  gives  the  complement  of  the  distribution  functions  of  the 


decision  axis. 
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F(zlN)  -  1-X  F(z]SN)  =  1-Y  (5.3) 

(2)  the  slope  of  a  differentiable  ROC  curve  is  equal  to  the  likelihood 
ratio  cut  value  corresponding  to  that  point. 

f=i=ez  (5.4) 

The  equation  for  the  pure  power  ROC  is 

X  =  YA  ,  A  >  1  (5.  5) 

To  determine  the  ROC  character  differentiate  Eq.  5.  5  with  respect 
to  X  to  obtain 


1  =  A  YA_1  ez  (5.6) 

Since  the  slope  of  the  ROC  curve  decreases  along  the  ROC  from  (0,  0) 
to  (1.  1).  Eq.  5.  6  can  be  used  to  determine  the  minimum  and  maximum 
values  of  z.  The  maximum  value  is  infinity  when  Y  is  zero,  and 
the  minimum  value  is  -In  A  when  Y  is  one.  Next,  solve  Eq.  5.6 
for  Y 


Y  = 


(5.  7) 


Differentiate  the  result  with  respect  to  z.  to  obtain  the  probability 
density  function  for  z  under  the  condition  SN. 
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The  distribution  function  for  the  decision  value  under  condition  N  is 
1-  X,  and  the  probability  density  for  the  decision  value  under  condi¬ 
tion  SN  is  the  derivative  of  1-Y  with  respect  to  the  decision  axis. 

P  (A)  =  /  [1-  X]M" 1  dY  (5.14) 

M  0 

When  the  number  of  pre  sentations  is  two,  Eq.  5.  14 
takes  on  a  very  simple  form 

1 

P0  =  f  (1-X)  dY  =  Area  under  the  ROC  (5.15) 

2  0 

Since  for  each  value  of  Y  .  the  length  of  the  horizontal  line  under 
the  ROC  curve  at  that  value  of  Y  is  1-X  ,  the  integral  yields 
the  area  under  the  ROC  curve.  Equation  5.  15  is  known  as  Green's 
theorem,  anu  will  be  discussed  in  Section  10.  1.  1.  For  the  power  ROC 
the  two- choice  probability.  Green's  theorem  value,  is  simply  evaluated 

P2(A)  -  /  [l-YA'dY  -  1  -  aTT  l5-16) 


To  obtain  an  expression  for  M  >  2,  factor  one  of  the  1-X  terms  out 
of  the  integrand  in  Eq.  5.  14 


/  (1-  YA]M  2  [1-  YA]  dY 

0 


(5.  17) 
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f(z :  sn) 


(5.  8) 


The  final  step  is  to  convert  the  probability  density  function  to  the  HOC 
character 


Forced  Choice  and  Second  Choice  Equations.  The  distributions  of  the 
decision-axis  variable  serve  as  a  model  for  many  performance  mea¬ 
sures  besides  theROC  curve.  The  following  situation  Is  used  in 


experimental  psychophysics  (Ref.  8,  pp. 30-36),  and  corresponds  also 
to  the  digital  communication  case  known  as  M-ary  coding  using  orthog¬ 
onal  waveforms.  The  description  will  use  the  psychophysical  ter¬ 
minology.  An  observation  consists  of  M  similar  portions.  The  SN 
condition  holds  for  one  and  only  one  of  these,  the  other  M  -  1  are  sta¬ 
tistically  independent  and  due  to  N.  When  the  SN  condition  is 
equally  probable  in  each  portion  of  the  observation  and  response 
utility  is  uniform,  the  experiment  is  considered  "symmetric”.  In  a 
symmetric  forced- choice  experiment,  the  observer  is  asked  to  indi¬ 
cate  his  principal  choice  as  to  the  portion  of  the  observation  due  to 
SN.  In  a  symmetric  second  choice  experiment  he  is  asked  to  indicate 
his  principal  and  second  choices. 

Equations  for  the  probability  of  these  choices  being 
correct,  for  an  observer  with  pure  power  ROC  curve  with  index  A, 
can  be  obtained  in  closed  form.  Let  P^(A)  denote  the  probability 
that  the  principal  choice  is  correct.  This  probability  will  be  a  func¬ 
tion  of  the  index  A.  Let  S^(A)  denote  the  probability  that  the  second 
choice  is  correct,  when  the  first  choice  is  wrong. 

If  the  principal  choice  is  correct,  then  the  part  of  the 
observation  due  to  condition  SN  yielded  a  higher  decision  value  than 
ail  of  the  M-l  values  due  to  condition  M. 

Pm(A)  =  Pr  fall  (M- 1)  values  of  fall  below  Zg^ j 


(5.  13) 


The  distribution  function  for  the  decision  value  under  condition  N  is 
1  -  X.  and  the  probability  density  for  the  decision  value  under  condi¬ 
tion  SN  is  the  derivative  of  1-Y  with  respect  to  the  decision  axis. 

P  (A)  =  /  fl  -  X]M" 1  dY  (5.14) 

m  o 


When  the  number  of  presentations  is  two,  Eq.  5.  14 
takes  on  a  very  simple  form 

1 

P 9  =  /  (1-X)  dY  =  Area  under  the  ROC  (5.15) 

Z  0 

Since  for  each  value  of  Y  ,  the  length  of  the  horizontal  line  under 
the  ROC  curve  at  that  value  of  Y  is  1  -  X  ,  the  integral  yields 
the  area  under  the  ROC  curve.  Equation  5.  15  is  known  as  Green's 
theorem,  and  will  be  discussed  in  Section  10.  1.  1.  For  the  power  ROC 
the  two- choice  probability.  Green's  theorem  value,  is  simply  evaluated 

P2(A)  -  ^  [l-YA|dY  -  1  -jJr  -  ATT  (516> 


To  obtain  an  expression  for  M  >  2,  factor  one  of  the  1-X  terms  out 
of  the  integrand  in  Eq.  5.  14 


1  \  m-2  a 

Pm(A)  =  /  U~  Y  j  [1-  Y  J  dY 


(5.  17) 


1  ■>  0 


pm:a) 


L  a  .  M-2 

/  [1-  Y  j  dY  - 


0 


ya[i- 


a  M-2 
YAJ  dY 


(5.  18) 


The  first  integral  is  simply  the  probability  that  the  principal  choice  is 
correct  had  there  been  one  fewer  alternative  .  The  second  integral 
is  evaluated  using  integration  by  parts.  The  general  formula  for  in¬ 
tegration  by  parts  is 

f  u  dv  =  uv  j  -  /  v  du 


where  the  bar  after  the  term  uv  indicates  that  this  product  is  evalua¬ 
ted  at  the  lower  and  upper  limits  as  are  the  integrals.  The  appropri¬ 
ate  choice  of  u  and  v  for  the  second  integral  in  Eq  5.  18  are  given 
in  Eq.  5.  19 


A  1  A  u 

u  =  Y  dv  =  -Y  [1-  Y  ]  dY 


.  M-  i  / 

du  =  dY  v  =  [1-YA]  /  A(M- 1) 


(5.  19) 


This  choice  is  partly  appropriate  because  the  value  of  u  at  the  lower 
limit  zero,  is  zero;  the  value  of  z  at  the  upper  limit  one,  is  zero. 
Therefore  the  term  uv  contributes  zero  to  the  integral. 


ya|  i-ya 


M-2 


dY  =  -  f  v  du  = 


0 


’ATM-7!) 


1  M- 1 

I  jl-Y  |  dY 

°  (5. 20) 


Equation  5.  20  substituted  in  (5.  18)  yields  a  recursion  relation  between 
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probabilities  of  the  principal  choice  being  correct  for  M  and  M-l. 


PM(A)  PM-1(A)  '  A(M-l)  PM(A^ 


(5.21) 


Collect  the  two  coefficients  of  P 


M 


1  + 


— — 1  P,.(A)  =•  P--  -(A) 
A(M-  1)J  Mv  M-l' 


(5.  22) 


and  rewrite 


PM(A)  ~  A  +  i/(M-l)  PM-1(A) 


(5.  23) 


Repeating  Eq.  5.  16 


p2<a>  “  TTi 


(5.  16) 


Applying  (5.  23)  once 


p  /Al  = _ — _ 

3  (A+1)(A  +  1/2) 


(5.  24) 


and  in  general 

AM 

PM^1(A)  '  (A  4 1)  (A  4  1/2)  ...  (A  4  1/M)  (5’  25' 

Each,  of  these  expressions  is  a  simple  algebraic  formula  which  takes 
on  the  value  1/M  for  A  =  1  (chance)  and  grows  monotonicaliy  to  one  as 
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A  approaches  infinity. 

The  second  choice  measure  is 

Sm(A)  -  Pr  [Second  choice  is  correct  i  First  choice  is  wrong]  (5.  26) 

A  conditional  probability  is  the  ratio  of  the  joint  probability  divided  by 
the  probability  of  the  condition.  The  probability  that  the  first  choice 
is  wrong  is  simply  one  minus  the  probability  that  the  first  choice  is 
right. 


Sm(A)  =  Pr  [Second  choice  is  correct  AND  First  choice  is  wrong]/ 

[1-Pm(A)]  (5.27) 

The  event  "secono  choice  is  correct  and  first  choice  is  wrong”  is  the 
event  that  one.  but  only  one,  of  the  N  based  decision  values  exceeds 
the  CN  based  decision  value. 

S^(A)  =  Prj(M-2)  values  of  fall  below  Zg^  and  one  z^.  exceeds  Zg^]/ 
ll-PM(A)j  (5.28) 

sm(A)  =  J  (M-l)  |l-X]M'2XdY/ll-PM(A)]  (5.29) 

where  the  multiplying  term  (M-l)  is  the  number  of  ways  of  choosing 
which  N  based  decision  value  exceeds  the  SN  based  decision 


value.  For  the  power  ROC  curve 


f 


i 
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1  a  M-2  a 

Sm(A)  =  (M- 1)  j  [1-  YA]  YA  dY/ [1-P  (A)]  (5.30) 


0 


This  integral  was  evaluated  previously,  Eq.  5.  20.  Therefore 


sm(a)  *  i 


M' 


(5.31) 


Using  Eq.  5.  25 


VA) 


.M-2 

_ A _ 

(A+1)(A+  ~)  ...  A+~Ti]  -AM_1 


(5.32) 


It  is  somewhat  simpler  to  evaluate  this  expression  for  small  values  of 
M  if  both  numerator  and  denominator  are  multiplied  by  (M  - 1)  fac¬ 
torial. 


O  I  A  \ 

M 


(M-  1) !  A 


M-2 


[(A+1)(2A+1)  ...  [(M-  1)  A+l]  j  -  (M-  1) !  A 


M-l 


(5.331 


If  there  were  an  experiment  with  only  two  presentations  and  the  signals 
had  to  be  in  one  of  them,  then  if  the  first  choice  is  incorrect,  the  sec¬ 
ond  choice  must  be  correct.  Equation  5.33  for  M  =  2  is 


S2<A>  *  ]a^Fa  *  1 


(5.  34) 


In  a  three- choice  experiment 


— - — 1 


~  !  , 
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S3(A>  =  3fh-  I  <  S3  <  I  (5'35) 

As  one  would  expect,  when  the  value  of  A  is  one  (chance)  the  second 
choice  probability  is  one- half.  (Since  the  observer  knows  his  first 
choice  was  wrong, he  is  guessing  between  two  intervals  with  no  basis 
for  decision. )  The  nonintuitive  result  is  that  as  the  ROC  curve  be¬ 
comes  better  ard  better  (A  approaches  infinity)  the  second  choice 
probability  does  not  approach  unity;  instead,  its  maximum  value  is 
two-thirds.  The  probability  of  the  principal  choice  being  right  does 
approach  one  when  the  ROC  index  approaches  infinity,  however,  the 
second  choice  probability  does  not  approach  one.  In  a  similar  manner, 
the  second  choice  in  a  four-choice  experiment  is  correct  with  proba¬ 
bility 


S4(A)  = 


6  A  2 _ 

11A2  +  6A  +  1 


(5.36) 


which  takes  on  the  chance  value  of  one-third  when  the  ROC  is  the 
chance  ROC.  but  takes  on  the  nonunity  limiting  value  of  6/ 1 1  when  the 
ROC  index  approaches  infinity. 

Although  a  great  many  equations  have  been  displayed 
here,  they  are  all  quite  simple  and  easily  obtained.  This  is  the  nice 
feature  of  an  algebraic  ROC  curve.  Such  computations  could  proceed 
ad  infinitum,  since  once  the  ROC  character  has  been  specified,  per¬ 
formance  of  the  decision  device  under  many  different  circumstances 
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can  be  obtained. 

5.2  Conic  ROC 

I’his  section  considers  a  somewhat  more  complicated 
algebraic  ROC  curve.  Since  the  formula  for  the  ROC  will  be  an  alge¬ 
braic  formula,the  same  general  procedure  for  determining  the  ROC 
character  from  the  equation  for  the  ROC  curves  shall  be  used.  Al¬ 
though  this  will  be  considerably  more  difficult  than  was  the  case  for 
the  pure  power  ROC,  the  work  is  generally  much  easier  than  when  the 
equation  for  the  ROC  is  not  algebraic.  The  purpose  of  this  section  is 
twofold:  (1)  To  demonstrate  again  the  relation  between  the  ROC  curve 
and  the  ROC  character  by  obtaining  the  character  from  the  curve, 
and  (2)  to  develop  the  three- parameter  class  of  conic  ROC  curves 
for  their  own  right. 

5  2.  1  From  Curve  to  Character.  The  general  tech¬ 
nique  for  obtaining  the  ROC  character  from  the  equation  for  the  ROC 
curve  has  basically  five  steps. 

(1)  Obtain  an  explicit  formula  for  y(x). 

(2)  Differentiate  this  expression  with  respect  to  x. 
substituting  ez  for  the  derivative  of  y  with 
respect  to  x. 

(3)  Invert  the  equation  to  find  x(z);  the  distribution 
function  for  z  under  condition  N  is  l-x(z). 

(4)  Differentiate  this  expression  with  respect  to  z 
to  obtain  the  probability  density  function  for  z 
under  condition  N. 
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5Z 

(5)  Multiply  this  density  function  by  e'  to  obtain 
the  ROC  character. 

The  conic  section  in  the  plane  is  the  locus  of  those 
points  which  can  be  described  by  a  second  order  algebraic  equation. 

Ay2  +  2Bxy  +  Cx‘  +  2Dy  +  2Ex  +  F  =  0  (5.37) 

A  regular  conic  section  ROC  curve  must  go  through  the  point  (0.0)  and 
the  point  (1.  1).  This  establishes  two  conditions  on  the  coefficients  in 
Eq.  5.37 

(0.  0)  F  =  0,  (1,  1)  =¥  A  +  2B  +  C  +  2D  +  2E  =  0  (5.38) 

It  is  convenient  to  use  the  first  condition  explicitly  but  to  reserve  the 
use  of  the  second  condition  until  the  end  of  the  development.  In  order 
to  solve  for  y  explicitly  in  terms  of  x.  rewrite  Eq*  5*  37  as  a  quad¬ 
ratic  equation  in  y 

Ay"  +-  2vBx  +  D)  y  +  (Cx2  -i  2Ex)  -  0  (5.  39) 

A  form  of  the  quadratic  equation  solution,  which  may  be 
unfamiliar  to  many,  is 

a  -i  0.  a£2  +  2b£  +  c  =  0  =  a£  -  -b  ±  ^b2  -  ac  (5.  40) 

This  is  used  in  the  present  development  because  it  eliminates  the  fac¬ 
tors  of  two  and  four  involved  in  the  more  common  equation  for  the 
solution  of  a  quadratic  equation.  For  the  moment  ',ssume  that  the 
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coefficient  A  is  nonzero  and  apply  the  standard  solution. 

A  *  0.  Ay  =  -Bx-D  ±  [B2x2  +  2B  Dx  +  D2  -  ACx2  -  2AEx]‘  5  (5.  41) 

Ay  =  -Bx-D  ±  [(B2  -  AC)  x2  +  2(BD  -  AE)  x  +  D2)'  5  (5.  42) 

Two  collections  of  terms  appear  in  Eq.  5.  42.  The  first  is  the  geo¬ 
metric  discriminant  B2  -  AC.  the  second  is  the  coefficient  of  x. 

Another  collection  of  terms  which  will  appear  later  in  the  development 
is  called  simply  K.  Let 

A  =  B2  -  AC,  T= •  RD-AE,  K  =  CD2  +  AE2  -  2BDE  (5.43) 

Using  these  new  symbols  Eq.  5.  42  becomes 

Ay  -  -Bx  -  D  ±  [Ax2  +  2rx  +  D2]' 5  (5.44) 

The  lirst  basic  step  is  complete,  an  expression  for  y  explicitly  in 
terms  of  x.  Differentiate  both  sides  of  the  equation  with  respect  to 
x  and  set  dv/dx  equal  to  e  . 

AeZ  =  -B  *  .  5|Ax2  +  2Tx  +  D2]"’  5[2Ax  +  2Tj  (5.45) 

The  third  basic  step  is  to  obtain  an  equation  for  x  explicitly  in  terms 
of  z.  Cancel  some  two’s  aad  move  B  to  the  other  side  of  the  equation. 


(Aez  +  B)  =  ± [Ax2  +  2rx  +  D2p  5  [Ax  +  T 


(5.  46) 
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Square  both  sides  and  multiply  through  by  the  denominator  of  the 
right-hand  side 

(Aez  B)2  |Ax2  +  2Tx  +  D2]  =  A2x2  +  2ATx  +  ^2  (5.47) 

The  resultant  expression  is  a  quadratic  equation  in  x.  Collect  the  co¬ 
efficients  of  the  various  powers  of  x. 

A  x2  [(Aez  +  B)2-a]  +  2Tx  [(Aez  +  B)2  ••  a] 

+  D2  (Aez  +  B)2  -  T-  =  0  (5.  48) 

The  large  factor  in  the  coefficient  for  both  x2  and  x  is  common. 

This  factor  may  be  zero;  however,  it  is  a  function  of  the  variable  z 
and  hence  will  be  zero  for  only  one  (or  two)  values  of  the  argument. 

We  shall  exclude  such  values  and  later  obtain  the  results  for  those 
values  bv  a  continuity  argument.  The  result  of  division  by  |(Ae  +B)"-A 
is 


A  x2  +  2Fx  + 


D2(AeZ  +  B)2  -  T2 
(Aez  +  B)2  -  A 


(5.  49) 


The  third  term  in  Eq.  5.  49  can  be  rewritten  to  obtain 


A  x2  +  2Tx  + 


D: 


(Ae 


D~A 

z 


-__r 

B)2 


=  0 


(5.  50) 


The  new  term  can  in1  simplified  by  utilizing  the  definitions  of  Eq.  5.  43. 
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D2A  -  F2  -  D2(B2  -  AC)  -  (B2D2  -  2ABDE  +  A2E2) 

D2A  -  r2  =  D2AC  h  2ABDE  -  A*E2  (5.  51) 

D2A  -  T2  =  A(CD2  -  2BDE  4- AE2)  -  -AK 

The  numerator  is  simply  -AK.  Next  examine  the  denominator, 

(AeZ  +  B)2  -  A  =  A2  e2z  +  2ABeZ  +  B2  -  (B2  -  AC) 

=  A  (Ae2z  +  2Bez  +  C)  (5.  52) 

The  denominator  also  contains  a  factor  A.  Utilizing  Eqs.  5.  51  and 
5.  52.  canceling  out  the  common  factor  of  A.  Eq.  5.  50  becomes 

A  x2  +  2Tx  +  jD2  -  K(Ae2z  i  2BeZ  +  C)_1]  =  0  (5.53) 

This  is  a  sufficiently  simple  form  to  apply  the  standard  quadratic  equa¬ 
tion.  (5.40).  and  obtain 

5 

A  x  =  -2T  ±  |r2  -  AD2  +  AK(Ae2z  r  2Bez  +  C)"1]  (5.  54) 

Use  Eq.  5.  51  again  t,)  simplify  the  term  T2  -  AD‘" 

O  r  -I  •  5 

Ax  -  -2T  ±  [AK  +  AK  (Ae  7  +  2BeZ  4  C)  |  (5.  55) 

This  completes  the  third  basic  step,  an  explicit  expression  for  x  in 
terms  vA  the  logarithm  of  the  likelihood  ratio  z.  Differentiate  with 
respect  to  z  to  obtain  the  probability  density  function  for  z  under 


the  condition  N. 
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-A  f(z  N' 


=  ±  .  5  VK  [A  +  A  (Ae2z  +  2BeZ  +  C)"  *]  A(-  1) 

_  2 

•  (Ae2z  +  2Bez  +  C)  (2Ae2z  +  2Bez) 


(5.  58) 


The  differentiation  has  been  accomplished  by  the  standard  chain  rule 
so  that  the  manipulations  may  be  traced.  Cleaning  up  the  minus  signs 
(5.  56)  becomes 


f(zlN)  =  VIC  (Ae2z  +  Bez)  (Ae2z  +  2Bez  +  C) 
•  [A  +  A(Ae2z  +  2Bez  +  Cf  *] 


(5.  57) 


This  expression  must  be  simplified  if  it  is  going  to  be  of  much  use. 

2  z 

Th*>  irsi  maneuver  is  to  take  a  square  root  of  the  factor  (Ae  +  .  .  . ) 
irom  vhe  third  factor  into  the  fourth  factor.  This  eliminates  the 
awkward  expression  containing  the  reciprocal  of  the  square  root  of 
\  t t  containing  another  reciprocal. 

9  - 1  5 

f(z.N)  -  yfK  eZ  (Aez  +  B)  (Ae  +  2Bez  +  C) 


•  [A(Ae2z 


2Bez  +  C)  +  A 


5 


(5.  58) 


The  last  factor,  under  the  square  root,  is 

A(Ae2z  +  2Bez  +  C)  +  A  =  A2e2z  +  2ABez  +  AC  +  B2  -  AC 

=  (Aez  +  B)2 


i 

! 


! 


i 

1 


(5.  59) 
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Eliminating  the  discriminant  A  yields  a  perfect  square,  and  nothing 
could  be  nicer  under  a  square  root  than  a  perfect  square.  The  den¬ 
sity  function  for  z  then  becomes  simply 

f(ziN)  -  VK  ez  (Ae2z  +  2BeZ  +  C)"  L  5  (5.60) 

The  fifth  basic  step  is  to  convert  to  the  ROC  character  by  multiplying 

by  e-  52 

life)  =  K'6  e1,5z  (Ae2z  +  2Bez  ,  C)  5  (5.61) 

Since  both  terms  are  raised  to  the  1.  5  power,  absorb  the  simple  ex¬ 
ponential  into  the  more  complicated  term. 

”1  5 

ff(z)  -  K‘ e  (AeZ  +  2B -<  Ce"z)  “  (5.62) 

Let  us  summarize  what  has  been  done.  A  regular  ROC 
curve  was  formed  by  passing  a  general  conic  through  the  point  (0,  0) 
and  the  po  nt  (1,  1).  The  fo^m  of  the  equation  was 

Ay?'  +  2Bxy  +  Cx“  4  2Dy  +  2Ex  =  0  (5.  37) 

with  the  side  condition 

A  +  2B  +  C  +  2D  +  2E  =  0  (5.  38) 

We  defined  two  particular  constants,  the  geometric  discriminant  and  a 


coefficient  K 
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A  =  B2  -  AC  K  =  CD2  +  AE2  -  2BDE  (5.43) 

After  straightforward  manipulation  the  formula  for  the  ROC  character 
was  obtained. 

_  *•  c 

7r(z)  =  K' 5  (AeZ  +  2B  +  Ce‘z)  °  (5.63) 

In  order  to  obtain  the  solution  the  assumption  was  made 
that  A  was  nonzero.  One  could  return  to  the  original  equation,  set 
A  equal  to  zero,  and  repeat  the  process.  Irdeed.  the  author  has  done 
just  that,  but  the  work  was  as  involved  as  that  above,  and  did  obtain 
the  same  solution.  Why?  A  continuity  argument  gives  the  answer. 

We  assume  that  if  A  is  zero  either  B  or  D  is  nonzero  other¬ 
wise  y  would  not  appear  in  (5.  37).  Consider  two  curves,  one  for 
A  zero  and  another  with  the  same  coefficients  but  with  A  nonzero. 
These  can  be  made  to  lie  arbitrarily  close  to  each  other.  The  ehar- 
amer,  Eq.  5.63,  is  also  a  continuous  function  of  A  when  either  B 
oi  C  is  nonzero.  We  may  therefore  omit  the  special  case  A  =  0 
from  our  derivation  and  fill  in  from  the  solution  by  this  continuity 
argument.  The  same  argument  holds  for  those  values  of  z  for  which 
the  expression  in  the  denominator  of  Eq.  5.  49  is  zero. 

5.  2,  2  Conic  ROC  Curves.  A  conic  has  six  original 
parameters.  Two  are  eliminated  by  the  regularity  conditions,  and 
one  more  could  be  eliminated  by  dividing  Eq.  5.  37  through  by  any  one 
of  the  constants  known  to  be  nonzero.  The  conics  :.re  essentially  a 


three- parameter  system  of  ROC  curves,  and  (basically)  three  parame¬ 
ters  appear  in  the  ROC  character,  A,  B,  C. 

The  coefficients,  A,  B,  C,  are  not  necessarily  the 
most  appropriate  coefficients  to  manipulate  in  detection  work.  A 
more  appropriate  set  of  parameters  might  be  based  on  the  range  of 
the  logarithm  of  the  likelihood  ratio.  Let  M  be  the  slope  of  the  ROC 
curve  as  it  approaches  the  point  (0,  0),  and  let  m  be  the  slope  of  the 
ROC  curve  as  it  approaches  (1. 1).  The  range  of  z  is 

In  m  <  z  <  In  M  (5.  64) 

M  -  +x  and/or  m  =  0  will  be  allowed.  Since  the  slope  of  the  chance 
diagonal  is  one,  the  value  for  m  is  less  than  one,  and  the  value  for 
M  is  greater  than  one  for  nonchance  regular  ROC  curves.  Given  an 
ROC  curve  in  an  expression  such  as  Eq.  5.  37,  these  terminal  values 
for  ROC  slope  can  be  obtained  from  Eq.  5.  45.  The  work  is  trivial 
but  tedious,  the  results  are  given  below. 

M  =  -E/D  m  =  -(B  +  C  +  E)/(A  +  B  +  D)  (5.65) 

In  Table  5.  1  are  displayed  parameters  for  various  conditions  of  ter¬ 
minal  slope.  Table  5.  1  also  contains  equations  for  the  ROC  curves 
and  characters.  The  fifth  case  in  Table  5.  1  is  the  symmetric  case, 
because  the  election  of  symmetric  bounds  on  this  type  of  ROC  charac¬ 
ter  means  that  the  ROC  curves  will  be  symmetric. 
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M  =  oc 

D  -  0  E  =  -l.  A  +  2B  +  C  =  2 
ROC:  Ay2  +  2Bxy  +  Cx2  -  2x  =  0 

_ )  5 

7T(z)  =  VA  (Aez  +  2B  +  Ce'z) 

K  =  VA,  T<*  ( 0 )  =  vA/8,  m  =  1  -  (A  +  B)  * 

I 

M  =  ac,  m  =  0 

B  =  1  -  A,  C  =  A,  D  =  0.  E  =  - 1.  A  -  1  -  2A 
ff(z)  =  VA,/ 8  (1  +  2A  sinh2  .  5z)  ^ 

ROC:  A(y-x)"  -  2x(l-y)  =  0 

| 

j  M  x.  _ mj=_0 

B  -  -(1  +  A).  C  =  2M  +  A.  D  =  1,  E  =  -M.  A  =  1  -  2A(M-1) 

ROC:  Ay*  -  2(1  +  A)  xy  +  (A  +  2M)  x2  +  2y  -  2Mx  -  0 

M  f-  oc. _ m  i  0 

D  =  1.  E  =  -M,  A  +  2B  +  C  =  2(M-  1) 

ROC:  Ay2  +  2Bxy  +  Cx2  +  2y  -  2Mx  =  0 

K  =  AM2  -  2MB  +  C,  m  -  1  -  (M-  1)(A  t  B  f  1) 

M  -  or.  tnM  -  1 

B  =  M  -  1  -  A.  C  =  A.  D  =  I.  E  =  -M;  A  ■  - 2M 1) 

K  =  A(M-l)2  t  2M(M-  1),  A  -  (M-  1)(M-  1-2A) 


Table  5.  1.  Canonical  conic  equations 
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Relevant  equations  and  parameter  values  are  collected 
in  Table  5.  2  for  the  case  of  zero  and  infinite  terminal  slopes. 


ROC  Curve 


Y  on 

A=C  A  B  neg.  diag. 


Chance  Diagonal  y  =  x  x  -x  -x 

Ellipse  (Long  Edge) 

Circle  y2  +  (1-x)2  =  1  1  -1  0 

Ellipse  (Pointed  End) 

Parabola  2(y-x)  =  1  -  (x+y- 1)2  .5  0  .5 

Hyperbola 

Singular  y  =  1  .0  11 


.  50 


.  707 


.  75 


1 


Table  5.  2.  Symmetric  conics,  unbounded  z 


When  the  ROC  is  symmetric  but  with  bounded  range  the  geometric 

•'  -  nes  and  the  corresponding  values  for  A.  B,  C.  D  and  A  are  displayed 

m  Table  5.3. 

Figures  5.  6  through  5.  12  display  on  linear  paper  the 
ROC  curves  and  the  ROC  characters  for  bounded  symmetric  conics. 

Each  figure  displays  the  complete  range  of  geometric  type  from  a 
chance  diagonal  to  the  Luce  ROC  curve,  all  with  the  same  terminal 
slopes. 


Chance  Diagonal 
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Two  ROC  characters,  one  concave  and  the  other  convex, 
are  compared  to  a  rectangular  character.  The  specific  cases  consid¬ 
ered  are  shown  in  Table  5.  4. 


(1) 

A  =  C  =  1,  B  = 

14, D  =  1,  E  =  - 

16,  K 

=  705,  A 

=  195,  T=  30 

ROC: 

y2  +  28  xy  +  x 

2  +  2y  -  32x  =  0 

tt(z)  =  (705)-  5 

(28  +  ez  +  e"z) 

■1.  5 

Izi  <  2. 

77 

(2) 

A  =  C  =  0,  B  = 

15,  D  =  1,  E  =  - 

16,  K 

=  480  A 

=  225,  f  =  15 

ROC: 

30xy  +  2y  -  32x  =  0 

ir(z)  =  (480)'  5 

(30)"  L  5 

izi  <  2. 

77 

A  =  C  =  - 1,  B  =  16,  D  =  1,  E  =  - 16,  K  =  255,  A  =  255,  T  =  0 
ROC:  -y2  +  32xy  -  x2  +  2y  -  32x  =  0 

S  <7  9.  -  I-  5 


tt(z)  =  (255)'  5  (32  -  ez  -  ez) 


!z!  <2.77 


Table  5.  4.  Collected  equations 


For  the  middle  case  the  coefficients  A  and  C  are 
zero.  Therefore  the  ROC  character  is  a  constant  over  its  full  range. 
The  range  has  been  chosen  to  correspond  to  a  maximum  slope  of  16 
and  a  minimum  slope  of  1/16.  For  Curve  1  and  3  the  z  range  is 
utilized  and  therefore  the  minimum  and  maximum  slopes  are  the  same. 
However  for  Curve  1,  A  and  C  are  plus  one,  while  for  Curve  3,  A 
and  C  are  minus  one.  This  means  that  as  the  parameter  z  increas¬ 
es  In  absolute  value  from  zero,  the  denominator  of  the  ROC  character 


Wi 


,  Vv* 


for  Case  1  will  increase,  and  the  ROC  character  will  decrease.  In 
just  the  opposite  manner  as  z  increases  in  absolute  value,  the 
denominator  for  the  ROC  character  for  Curve  3  will  decrease  in  mag¬ 
nitude  and  the  ROC  character  will  increase  for  large  z.  The  ROC 
characters  and  the  corresponding  ROC  curves  are  plotted  on  linear 
paper  in  Fig.  5.  13.  The  cusp  nature  of  the  third  ROC  character  con¬ 
centrates  more  probability  at  extreme  values  of  z,  and  therefore  places 
more  emphasis  on  those  arcs  of  the  ROC  with  extreme  slope.  This 
means  that  it  spends  more  "time”  rifsing  sharply  from  the  origin  than 
does  Curve  2  with  a  flat  ROC  character;  the  ROC  curve  is  superior  to 
the  rectangular  hyperbola  with  the  same  limiting  slopes.  In  contrast¬ 
ing  manner,  the  first  ROC  character  has  the  more  normal  convex 
appearance  with  mode  at  z  =  0.  This  emphasizes  slope  in  the  neigh¬ 
borhood  of  one  (z  =  0).  If  one  were  to  use  an  approximation  procedure 
for  data  by  using  rectangular  hyperbola  for  the  first  order  approxima¬ 
tion,  it  seems  fairly  evident  how  one  might  adjust  the  ROC  character 
compared  to  the  fiat  character  to  obtain  better  fits  in  the  neighborhood 
of  the  negative  diagonal. 

The  three  ROC  characters  of  Table  5. 4  and  their  ROC 
curves,  can  be  compared  to  normal  by  plotting  the  curves  on  normal- 
normal  paper.  This  is  done  in  Fig.  5.  14.  These  hyperbolas  differ 
from  normal  ROC  curves  by  indicating  higher  detection  indices,  d, 
along  the  negative  diagonal  and  lower  values  off  the  diagonal.  That  is, 
they  have  sharper  "elbows"  along  the  negative  diagonal.  For 


comparison,  a  normal  curve  with  d  =  2.  3104  has  been  chosen  to 
approximate  Curve  1  in  the  .05  to  .50  false  alarm  probability  re¬ 
gion.  The  character  and  ROC  curve  for  the  first  hyperbola  have  been 
repeated  in  Fig.  5.  15  on  linear  paper,  along  with  the  approximating 
normal  curve  and  character.  The  normal  ROC  character  extends  to 
plus  infinity,  therefore  we  would  expect  it  to  have  far  superior  per¬ 
formance  at  very  low  false  alarm  rates.  Indeed,  for  x=.01  the 
normal  ROC  has  y  =  .  21  while  the  hyperbolic  ROC  shown  has  only 
y  =  .  132.  This  disc  'pancy  might  be  exceedingly  important  in  some 
situations.  If  the  data  for  which  an  approximating  ROC  curve  is  de¬ 
sired  are  based  on  fewer  than  500  Observations,  then  determination  of 
an  accurate  false  alarm  rate  of  the  order  of  .  01  would  not  be  possible, 
and  this  region  of  the  ROC  would  be  of  little  interest.  A  feature  of  the 
chosen  tit  is  that  the  one  percent  false  alarm  rates  occur  at  roughly 
the  same  z  value.  As  can  be  seen  frc.i  Fig.  5.  15,  from  the  maxi¬ 
mum  z  value  of  the  conic  ROC  dov/n  to  about  z  =  1  the  ROC  charac¬ 
ter  for  the  conic  is  larger  than  the  character  of  the  normal;  the  conic 
ROC  rises  proportionately  faster.  By  the  time  the  false  alarm  rate  is 
ten  percent  the  two  curves  cross.  If  one’s  definition  of  ’’close"  means 
"as  points  appear  when  plotted  on  linear  paper. "  then  these  two  ROC 
characters  yield  ROC  curves  which  are  quite  close. 

5.3  Lor- Lor  Paper 

The  previous  section  developed  the  three  parameter 
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ir(z) 

4-, 


Jr-J - i - 1 - 1 _ i  i 

3-2-10  12 


Z 

Fig.  5. 15  Comparison  of  curves  and  characters 
for  a  conic  and  a  normal  ROC 
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class  of  regular  conic  ROC  curves.  If  one  wishes  to  use  this  form  of 
ROC  curve  and  character  to  fit  data,  one  would  appreciate  some  graph¬ 
ical  aid  to  obtaining  the  fit.  Those  who  have  worked  with  normal  ROC 
approximations  are  familiar  with  the  usefulness  of  normal- normal 
paper.  In  some  experimental  work  thedata  have  considerable  scatter, 
and  the  availability  of  various  types  of  special  paper,  a  straight-edge, 
and  the  experimenter’s  eye  can  be  as  powerful  as  a  digital  computer. 

This  section  will  develop  the  formulae  for  the  coordi¬ 
nates  of  special  paper,  called  lor- lor  paper  on  which  the  rectangu¬ 
lar  hyperbolas  plot  as  straight  lines  with  slope  one.  This  is  followed 
with  the  general  equation  for  the  symmetric  conic  ROC  curves  as  they 
would  plot  on  this  paper.  Finally,  graphs  are  presented  of  the  hyper¬ 
bolas.  other  conics,  and  normal  ROC  curves,  on  lor- lor  paper.  The 
coordinate  transformation  which  shall  be  used  was  used  by  R.  A.  Fisher 
in  his  study  of  the  correlation  coefficient,  and  is  sometimes  known  as 
the  Fisher-z  transformation.  The  coordinate  transformation  has  also 
been  used  in  the  study  of  sequential  observation  (Ref.  25).  I  am  un¬ 
aware  of  the  commercial  availability  of  such  paper. 

The  ratio  of  the  probability  of  an  event  occurring  to  the 
probability  of  that  same  event  not  occurring  is  sometimes  called  the 
"odds  ratio.  ”  Quite  naturally,  the  logarithm  of  this  ratio  is  leferred 
to  as  the  logarithm  of  the  odds  ratio,  or  simply  log- odds- ratio.  We 
shall  reserve  the  symbol  L  for  the  log- odds- ratio  transformation 
(simply,  che  lor  transformation). 


Fig.  5.16.  Comparison 


Definition  of  log- odds- ratio.  L(  ): 


0  <  P  <  1,  L(P)  =  In  (5.  66) 

For  very  small  values  of  its  argument,  the  log-odds- 
ratio  is  very  close  to  being  a  simple  logarithm.  To  demonstrate  how 
close,  the  log- odds- ratio  has  been  plotted  against  probability  on  semi¬ 
log  paper  in  Fig.  5.  16.  For  values  of  probability  below  something  like 
three  percent  to  ten  percent  L(P)  is  graphically  equivalent  to  In  P. 

L  is  an  odd  symmetric  function  about  50  percent  probability.  This 
may  be  seen  by  substituting  (1-P)  for  P  in  Eq.  5.66.  Therefore 
when  P  becomes  very  close  to  unity,  L(P)  will  again  become  nearly 
logarithmic,  but  with  the  logarithm  of  (1-  P). 

L(l-P)  =  -L{  P) 

P  -  0.  L(P)  -  In  P  (5.67) 

P  -  1,  L(P)  —  -ln(l-P) 

This  symmetry  property  makes  the  lor  scale  more  useful  than  the 
ordinary  log  scale  for  plotting  ROC  curves.  In  Fig.  5.  17  the  log.  lor  , 
and  normal  scales  are  displayed.  The  placement  was  chosen  to  align 
the  one  percent  and  fifty  percent  points  on  the  scales,  fne  major  con¬ 
trast  between  the  lor  and  log  scales  is  in  the  symmetry.  The  log 
scale  expands  the  region  below  .  50,  but  greatly  compresses  the  re¬ 
gion  between  .  50  and  1. 00,  while  the  lor  scale  expands  both  regions 


5.  17.  Comparison  of  lor,  normal  and  log  scales 


symmetrically.  The  normal  scale  is  also  a  symmetric  scale  about 
.  50  probability.  On  ail  three  scales  the  marked  positions  for  proba¬ 
bility  are  not  uniformly  spaced.  The  scale  variables,  L,  t,  and  s, 
are  uniformly  spaced  on  the  axes. 

The  quantity  dr  advanced  by  Tanner,  Swets,  and  Green 
as  an  index  of  detectability  is  directly  related  to  the  normal  scale  when 
the  experimental  procedure  is  a  YES- NO  test  (Ref.  13,  10).  For  each 
point  (x,  y)  on  the  ROC  curve  the  detection  index  is  given  by  Eq.  5.  68. 

df  =  f  1(”)  -  f  *(x)  (5.  68) 

The  function  3>  is  the  normal  distribution  function  and  the  symbol 
$  1  indicates  the  inverse  function.  The  inverse  of  the  normal  proba¬ 
bility  distribution  function  is  the  linear  distance  scale  on  normal- 
normal  paper.  Therefore  the  value  of  dT  for  any  ROC  point  is  its 
vertical  scale  value  minus  the  horizontal  scale  value  on  normal- 
normal  graph  paper.  The  normal  ROC  curves  plot  as  straight  lines 
with  a  slope  of  one  or.  normal- normal  paper.  The  difference  between 
the  vertical  and  horizontal  scale  values  is  constant  over  the  normal 
ROC.  The  numerical  value  of  the  constant  is  the  square  root  of  the 
index  d  appearing  in  the  ROC  character.  Therefore,  for  the  normal 
ROC  the  value  of  d*  is  the  square  root  of  d  over  the  whole  ROC  curve. 


Normal  ROC:  d'  =  Vd 


(5. 69) 
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In  an  analogous  manner  we  shall  define  a  lor  index  of  detectability 
of  a  point  as  the  difference  between  the  vertical  scale  coordinate  and 
the  horizontal  scale  coordinate  on  lor-lor  graph  paper. 

Definition:  M'  =  L(y)  -  L(x)  (5‘  70> 

The  coefficients  of  the  equation  for  the  regular  rectan¬ 
gular  hyperbola  are  repeated  in  Eq.  5.  71. 

Regular  Rt.  Hyperbola:  A  =  0,  B  =  M-l,  C  =  0,  D  =  1,  E  =  -M  (5.  71) 

M  is  the  maximum  slope  of  the  ROC  curve,  and  the  reciprocal  of  the 
minimum  slope  of  the  ROC  curve.  Substitute  these  values  in  the  gen¬ 
eral  equation,  and  divide  through  by  a  common  factor  of  two. 

(M-l)  xy  +  y  -  Mx  =  0  (5-  72) 

Such  an  equation  can  be  explicitly  solved  for  y.  The  solution  is 

nr  _  Mx  __  (5.73) 

y  Mx  +  1  -  x 


Subtract  y  from  one 

i  _  1  -  x 

1  "  y  Mx  +  1  -  x 


and  take  the  ratio  of  y  to  (1-  y). 


(5.74) 


(5.  75) 
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In  logarithmic  form, 

In  r-£-  -  In  -=■— —  +  In  M  (5.  76) 

1-y  1-x 

Fquation  5.  76  can  be  simply  rewritten  in  terms  of  the  lor  scale. 

L{y)  =  L(x)  +  In  M  (5.  77) 

If  graph  paper  is  used  so  that  the  true  linear  scale  along  the  two  axes 
is  the  lor  of  the  probability  marked  on  those  axes,  then  a  regular 
rectangular  hyperbola  will  plot  as  a  straight  line  with  slope  of  one. 
Along  the  entire  ROC  curve  the  lor  index  of  detectability,  M\  will 
be  constant  and  equal  to  the  logarithm  of  M. 

Rt.  Hyp.  ROC:  M'  =  In  M  (5.78) 

In  Fig.  5.  18  curves  of  constant  M'  are  plotted  on  lor- lor  paper. 

A  subtle  distinction  which  is  very  necessary  is  the  dif¬ 
ference  between  M  and  M\  The  same  difference  must  be  made  be¬ 
tween  d  and  d'  (Ref,  10).  The  index  d  applies  only  to  the  normal 
ROC  curve,  and  the  normal  ROC  character.  The  index  M  applies 
only  to  the  regular  rectangular  hyperbola  ROC  curve,  the  rectangular 
ROC  character.  The  indices  d*  and  M'  apply  to  any  point  on  the 
ROC  graph.  In  general,  the  indices  d*  and  M'  will  vary  along  a 
given  ROC  curve.  Only  when  the  ROC  curves  are  the  special  cases  of 
normal  (for  d’)  and  rectangular  hyperbola  (for  M’)  will  these  indices 
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5.18.  Lor- Lor  paper  with  lines  of  equal  M' 
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be  constant  over  the  entire  RCC  curve,  and  directly  related  to  the 
parameter  for  the  entire  curve. 

In  Fig.  5.  19,  a  family  of  normal  ROC  curves  are 
plotted  on  lor- lor  paper.  The  index  M'  varies  along  each  curve, 
although  not  radically.  Along  the  negative  diagonal,  the  value  for 
M'  is  a  minimum,  and  M'  increases  along  a  given  curve  as  the 
point  moves  away  from  the  negative  diagonal.  This  indicates  that 
the  normal  character  concentrates  more  probability  at  large  magni¬ 
tude  z  values  than  would  a  rectangular  character,  which  yielded  the 
same  negative  diagonal  point. 

When  working  with  ROC  curves  with  nonzero  minimum 
and  noninfinite  maximum  slope,  then  one  may  obtain  information 
about  the  ROC  character  from  the  ROC  curve  plotted  on  lor- lor 
paper.  In  order  to  demonstrate  this  interpretation  procedure,  a  set 
of  conic  ROC  curves  with  the  same  initial  and  slope  values  were  se¬ 
lected.  The  parameters  for  these  are  given  in  Table  5.  5. 


Ne;.  Diag. 
Point 

A 

B 

C 

D 

E 

A 

(.30.  .70) 

5i 

3  1 

6  8 

C  1 

58 

1 

-10 

-11.25 

(.2405.  .7595) 

0 

9 

0 

1 

-10 

81. 

(.20.  .80) 

10  3 

_13 

1 

-10 

105 

(.  10.  .  90) 

-2.21875 

11.21875 

-2.21875 

1 

-10 

120. 94 

Table  5.  5.  Table  of  parameters 


Each  of  these  curves  has  maximum  slope  of  ten  and  minimum  slope 
of  .  1.  A  rectangular  hyperbola  with  index  M’  =  log  10  =  2.  30  also 
has  these  same  terminal  slope  values.  These  ROC  curves  are  shown 
on  linear  paper  in  Fig.  5.  20.  The  Luce  ROC  curve  with  slopes  10  and 
.  1  is  also  drawn.  This  is  thd  limiting  value  for  conic  ROC  curves 
with  these  slopes.  The  hyperbola  going  through  the  negative  diagonal 
point  (.  10, .  90)  lies  very  close  to  the  Luce  ROC  curve.  The  hyperbola 
going  through  the  negative  diagonal  point  (.  20, .  80)  is  superior  to  the 
rectangular  hyperbola,  and  the  elipse  going  through  the  negative  dia¬ 
gonal  point  (.  30, .  70)  is  inferior  to  the  rectangular  hyperbola  ROC 
curve. 

Figure  5.  21  displays  the  ROC  characters  for  the  same 
curves  on  semi- log  paper.  Figure  5.  22  displays  the  ROC  curves  on 
lor- lor  paper.  The  rectangular  hyperbola  has  flat  character,  and 
plots  as  a  straight  line  on  Fig.  5.  22 .  Since  all  of  the  curves  have 
the  same  minimum  and  maximum  slope  values,  the  z  range  is  com¬ 
mon.  This  is  shown  as  a  common  range  in  Fig.  5.  21  from  -2.  3  to 
+2.  3.  This  appears  on  the  ROC  curve  in  Fig.  5.  22  as  the  common 
asymptotic  M'  value.  All  of  the  curves  are  asymptotic  to  M'  =  2. 30. 
The  curve  going  through  the  negative  diagonal  point  (.  30, .  70)  was  in¬ 
ferior  to  the  rectangular  hyperbola,  while  the  other  two  were  superior. 
The  same  relationship  holds  for  large  magnitude  z  values  on  the  ROC 
character.  Hie  (.  30, .  70)  curve  has  lower  char^  f'ter  value  than  the 
flat  character  while  the  other  two  have  higher  character  value  at 
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large  magnitude  z.  Of  course,  just  the  opposite  relation  holds  when 
z  is  close  to  zero.  Since  the  ROC  curve  is  obtained  by  multiplying 
the  ROC  character  by  e*'  ^z,  and  integrating,  the  effect  of  the  ROC 
character  for  large  magnitudes  of  z  overshadows  the  effect  at  small 
magnitudes  of  z.  The  ROC  character  for  the  (.  10, .  90)  case  rises 
from  very  small  values  to  the  value  .16  at  z  =  ±2.  2,  to  the  value 
112.  4  at  z  =  ±2.  3.  If  the  peak  character  value  of  112  were  main¬ 
tained  constant  over  a  z  range  of  only  0026,  it  would  accumulate 
an  area  of  .  288.  The  integrated  ROC  character  for  the  Luce  ROC  has 
jumps  of  .  288  at  z  =  ±M';  that  is,  the  character  has  delta  functions 
of  magnitude  .  283  at  ±M'.  The  two  ROC  curves  and  characters  are 
close. 

The  above  example  compared  symmetric  conic  ROC 
with  the  same  limiting  slope  values.  The  next  example  compares  non- 
symmetric  conic  ROC's  with  the  same  limiting  slope  values.  The  limi¬ 
ting  slopes  chosen  are  a  maximum  slope  of  100  and  a  minimum  slope 
of  .1.  This  means  that  the  z  range  will  be  from  -2.3  to  +4.6. 

The  coefficients  for  the  five  cases  considered  are  given  in  Table  5.  6. 
The  ROC  curves  are  displayed  on  linear  coordinate  in  Fig.  5. 23. 

Since  the  initial  and  final  slope  values  are  not  reciprocal  values,  there 
is  no  rectangular  hyperbola  with  constant  M*  value  to  compare  with 
these.  In  Fig.  5. 23  are  shown  portions  of  two  constant  M'  curves. 

In  the  high  slope  region,  to  the  left  of  the  negative  diagonal,  is  drawn 
the  rectangular  hyperbola  with  index  M*  =  4. 6.  To  the  right  of  the 
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(J-  JO) 
4r  lr 

y  -  lOOx  = 

°.  x  <  ni 

> 

lOy-x  =  9 

x  > 

Luce  ROC 

Table  5.0.  Table  of  coefficients 

negative  diagonal,  in  the  low  slope  region,  is  drawn  the  rectangular 
hyperbola  with  M*  =  2.3.  In  the  neighborhood  of  the  corners  points 
of  the  ROC  all  of  the  curves  will  come  together  because  they  approach 
the  corners  with  the  terminal  slope  values. 

The  corresponding  ROC  characters  are  displayed  on 
semi-log  paper  in  Fig.  5.  24.  The  ROC  curves  on  lor-lor  paper  are 
displayed  on  Fig.  5.  25.  The  two  curves  which  are  inferior  to  both  of 
the  constant  M*  curves  display  the  same  general  peak  character  as  in 
the  previous  example.  The  curve  marked  (.  1, .  9)  hugs  the  Luce  ROC 
for  the  low  slope  values,  but  not  for  high  slope  values.  Therefore  the 
ROC  character  displays  a  nearly  delta- like  behavior  at  the  minimum 
z  value,  -2. 3,  but  not  nearly  so  sharp  a  cusp  at  the  maximum  z 
value,  4. 6.  The  curve  marked  (.  2, .  8)  has  a  monotone  decreasing 
ROC  character.  The  value  of  this  character  fells  between  that  of  the 
two  limiting  constant  M*  values  between  z  ~  - 1. 4  and  z  =  2. 6.  This 
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Fig.  5.24.  ROC  characters  -2.3  <  z<  +4.6 
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corresponds  to  the  range  of  slopes  from  .  25  to  13.  5.  Since  the 
ROC  character  for  (.  2, .  8)  falls  below  the  limiting  M’  character  at 
z  =  4.  6,  the  ROC  curve  will  be  inferior  to  the  M*  =  4.  6  level  near 
the  origin.  Since  the  ROC  character  for  (.  2, .  8)  is  larger  than  that 
for  M’  =  2.  3,  the  ROC  carve  will  fall  above  the  M'  =  2.  3  value  as 
the  curve  approaches  (1,  1). 

In  Fig.  5.  26,  a  pure  power  law  ROC  is  plotted  on  lor- 
lor  paper.  Two  asymptotes  are  drawn  to  this  curve.  The  left-hand 
asymptote  is  obtained  from  the  logarithmic  nature  of  the  lor  for  small 
values  of  argument.  Since  the  pure  power  ROC  plots  as  a  straight  line 
on  log- log  paper,  with  slope  A  (in  this  case,  2)  the  pure  power  ROC 
will  plot  as  a  straight  line  with  slope  A  on  lor- lor  paper  when  both 
x  and  y  are  small.  Since  the  pure  power  ROC  curves  approach  the 
(1,  1)  point  with  slope  1  'A,  the  ROC  curve  will  become  asymptotic  to 
the  rectangular  hyperbola  with  index  M'  =  In  A. 

5.  4  Choosing  a  Fitting  Conic 

The  conic  equation  contains  six  coefficients,  A,  B,  C, 

D.  E,  and  F.  However,  restriction  to  regular  ROC  curves  places 
three  side  conditions  on  these  coefficients,  leaving  three  degrees  of 
freedom.  It  would  be  desirable  to  have  a  procedure  to  obtain  the 
"best”  fitting  ROC  to  any  amount  of  data.  Such  a  procedure  has  not 
been  developed  in  this  present  work. 

If  three  representative  points  are  selected,  the  procedure 
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to  fit  the  regular  conic  through  these  three  points  is  fairly  simple 
and  straightforward.  One  must  first  check  that  the  broken  line  seg¬ 
ment  that  goes  from  the  point  (0,  0)  to  the  point  with  lowest  false 
ala^-m  rate,  then  to  the  point  with  next  false  alarm  rate,  to  the  point 
v/ith  the  highest  false  alarm  rate,  and  then  to  the  upper  corner  (1, 1), 
is  convex  and  that  no  three  points  are  coliinear..  This  last  means  that 
there  are  really  four  distinct  straight  line  segments  going  from  (0, 0) 
to  (1,  1).  The  only  problem  is  to  determine  the  coefficients  so  that 
Eq.  5.79  is  satisfied  for  these  three  points. 

Ay^2  +  2Bx.y.  +  Cx^2  +  2Dy^  +  2Ex^  =  0  ,  i  =  1,2,3  (5.79) 

Utilizing  the  equation  that  makes  the  ROC  go  through  (1, 1),  and  mo¬ 
mentarily  making  the  assumption  that  E  is  nonzero,  the  set  of 
equations  (5.  79)  becomes  Eq.  5.  80. 


Ay.(l-y.)  +  By.(l-x.)  +  C(y.-x.2)  =  (y.-x.)  I  =  1,2,3 
E  -  -1  D  =  1  -  B  -  .  5(A+C) 


(5.  80) 


If  the  determinant  of  the  three  equations  in  three  variables  is  not 
zero,  then  the  assumption  that  E  was  nonzero  is  valid  and  the  ROC 
has  been  determined.  If  the  determinant  is  zero,  we  may  assume 
that  E  =  0  and  then  resolve  the  equations  in  the  first  part  of  Eq.  5.  80 
aft<  r  setting  the  right-hand  side  to  zero. 

At  a  meeting  of  the  Acoustical  Society  of  America  in 
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1965,  students  of  Lloyd  Jeffries  presented  some  ROC  curves  with 
very  distinct  and  measurable  slopes  from  the  origin  and  the  (1, 1) 
point.  If  one  wishes  to  match  a  conic  ROC  to  such  data,  these  two 
slopes  and  any  one  point  are  sufficient  to  define  the  conic.  The  only 
restriction  is  that  the  point  must  be  chosen  to  lie  above  the  chance 
diagonal  and  interior  to  the  Luce  ROC  curve  with  the  given  slopes. 
The  Luce  ROC  curve  with  these  given  slopes  is  given  by 


0  <  m  <  1  <  M, 


y  =  Mx  C<x<^ 
y=  1-mtmx  HTS<*<  1 


(5.  81) 


A  computer  program  was  written  to  do  this  matching  and  is  contained 
in  Appendix  E .  The  input  data  for  such  a  program  are  the  two  slopes 
and  the  additional  point.  The  program  is  designed  to  check  that  the 
point  lies  above  the  chance  diagonal  and  below  the  Luce  ROC  curve,  to 
determine  the  coefficients  of  the  equations,  and  in  addition  to  print  out 
the  ROC  curve  in  linear  ard  lor  coordinates  and  determine  the  ROC 
character  for  the  resultant  conic. 


CHAPTER  VI 


CHARACTER  FITTING  BY  TRUNCATED  NORMAL  CHARACTER 


6, 1  Metastatic  Normal 

The  purpose  of  this  chapter  is  to  present  various  methods 
of  approximating  ROC  characters  by  sections  of  normal  character,  in 
order  to  yield  approximate  ROC  curves  through  the  use  of  normal 
tables.  Each  technique  assumes  £.  certain  limited  amount  of  knowledge 
about  the  true  ROC  character,  and  based  on  this  information  achieves 
some  sort  of  fit  in  terms  of  a  truncated  normal  character. 


The  original  ROC  curve  is  subscripted  by  a  one  and  the  metastatic 
derivative  subscripted  by  a  two.  Two  expressions  that  repeat  in 
these  equations  are  the  range  of  the  X  axis  that  is  mapped  onto  the 
metastatic  derivative,  and  the  portion  of  the  Y  axis  that  is  mapped 
onto  the  metastatic  derivative.  It  will  be  convenient  to  have  single 
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letters  for  these  positive  quantities.  Let 


KN  =  b  -  a  KgN  =  Yj(b)  -  Yjia)  (6. 1) 


Two  of  the  salient  characteristics  of  an  ROC  character 
are  the  minimum  and  maximum  values  of  z.  Let  the  maximum  value 
be  z  and  the  minimum  value  be  z,  .  The  subscript  was  chosen  to  cor- 

cl  D 

respond  to  the  values  on  the  original  ROC  curve. 

z  is  value  of  z„  when  X.  =  a,  Y.  =  Y.(a) 
a  1  ill  (6. 2) 

z^  is  value  of  z ^  when  X^  =  b,  Y^  =  Y^(b) 


It  will  also  be  convenient  to  have  a  notation  for  the  shift  between  the 
z  values  on  the  first  (the  original)  and  the  second  (the  metastatic) 
decision  axes.  This  is  the  logarithm  of  the  ratio  of  and  KgN, 
herein  denoted  by  Zq. 


=  ln  VW 

(6.  4) 

II 

N 

to 

1 

N 

O 

(6.  5) 

This  chapter  is  devoted  to  the  metastatic  normal  ROC 
curve.  Therefore  x^z)  will  always  be  assumed  to  be  a  normal  ROC 
character.  The  general  form  of  the  ROC  character  with  index  d  is 

_  d  _  z_ 

rN(z)  * e  8  iht  e  2d  <6- 6) 

In  order  to  avoid  confusion  between  the  normal  detection  index  d,  and 


the  parameter  in  an  original  ROC  character  which  carries  over  as  a 
parameter  of  the  metastic  character,  write  m2  for  &  The  notation 
that  0  is  the  normal  probability  density  function  in  standardized  form 
and  $  is  the  normal  distribution  function  is  as  used  throughout  this 
work.  Rewriting  the  normal  ROC  character  with  these  changes, 


JN(zl' 


The  ROC  character  for  a  metastatic  normal  ROC  curve  is 


(6.7) 


”MN(z)  =  mVKN  Ksn  e 


z,  <  z  <  z 
b  a 


(6.  8) 


In  Fig.  6.  1  are  sketched  two  complete  normal  density 


lunctions  on  the  original  z^  axis,  and  underneath  this  the  z ^  axis.  The 


Fig.  6. 1.  Sketch  of  two  z  axes 
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three  key  z ^  values  are  the  minimum  value,  z^,  the  mode  value,  Zq, 
and  the  maximum  value,  z  .  Since  the  mean  of  the  N  distribution  is 
at  5m2  the  values  for  the  false  alarm  probabilities,  Xp  can  be 
written  in  terms  of  the  normal  distribution  function  as  given  below. 


Xjfz,)  =  1-4.  (-1  ^-5m  )=  1  •  4>(^-  +  .  5m)  (6.  9) 


At  the  extremes  of  the  z ^  range, 


a  =  1  -  $  +  •  5m) 


(: 


r  2  —  2  V 

b  =  1  -  $  ( -1—0.  +  .  5m }  (6.  10) 


m 


/ 


Becausethe  mean  of  the  SN  distribution  is +.  5m2  ,  the  formula  for  the 
probabilities  will  be  quite  similar  to  Eq.  6.  9. 


Y^)  =  1  -  $ 


(6. 11) 


The  areas  remaining  under  the  normal  density  curves  inside  the  abrupt 
truncation  point  are  and  , 


The  general  equation  for  the  metastatic  ROC,  Eq.  3.  37,  becomes  the 
following  fo.-  the  metastatic  normal  ROC. 
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(6. 15) 

In  the  following  sections,  the  problem  of  fitting  a  given 


ROC  character  with  the  metastatic  normal  ROC  character  of  Eq.  6.  8 
is  attacked.  The  metastatic  normal  is  given  by  three  parameters. 

Given  a  specific  I'OC  character,  three  salient  characteristics  must  be 
selected  to  determine  the  matching  metastic  normal.  It  is  assumed 
that  the  ROC  character  to  be  fit  contains  a  simple  mode  and  has  the  general 
appearance  of  being  normal  about  this  mode.  This  can  be  most  easily 
seen  by  looking  at  the  logarithm  of  the  ROC  character,  since  the  log¬ 
arithm  of  the  normal  character  is  a  simple  parabola.  It  is  also 
assumed  that  the  character  to  be  fit  covers  a  bounded  z-range, 

6.  2  First  Procedure 

In  this  section  the  equations  necessary  to  fit  a  meta¬ 
static  normal  ROC  character  in  a  given  ROC  character  are  based  on 
the  minimum  value  and  mode  value  of  that  given  character,  and  the 
ratio  of  the  character  values  at  the  mode  and  the  minimum.  It  will  be 
assumed  that  the  maximum  value  of  z  is  above  the  mode,  but  no  other 
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use  will  De  made  of  the  maximum  value.  This  information  is  summarized 
in  Eq.  6.  16. 

Known:  zb,zQ,  that  z&  >  zQ,  ff(zQ)/ tt( zfe)  =  rfa  >  0  (6. 16) 

This  procedure  is  based  on  only  the  information  given 
in  Eq.  6. 16.  The  metastatic  normal  ROC  character,  Eq.  6.  8,  evaluated 
at  the  mode,  is 


WV  * 


K, 


SN 


m 

8 

e  717 


(6.  17) 


The  metastatic  normal  ROC  character  differs  from  this  value  by  only 

an  exponential  term.  Equation  6.  17  can  be  used  to  simplify  the  writing 

of  the  off-mode  values  of  7rx_„.  The  known  information  relates  the 

NM 


character  values  at  the  mode  and  at  the  minimum  value, 


77MN(zb)  6 


(zb  ~  v 

2  m2 


(6.  18) 


Use  this  equation  to  determine  the  value  for  the  parameter  m 


m  = 


(6. 19) 


The  detection  index,  m2  ,  of  the  origi  x.  lal  i  OC  curve,  has  been 
determined.  The  lower  z^  cut  point  z^  -  z^  was  contained  in  the 
original  information.  To  determine  the  upper  Zj  cut  point,  zft  -  Zq, 
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requires  a  devious  route  which  is  not  always  successful.  The  only 
clue  to  the  value  of  the  upper  cut  point  lies  in  the  normalizing  constant 
and  K  .  Rewriting  Eq.  6.  4  in  an  equivalent  form 

r 

e  Sn  =  20^ 

the  task  of  obtaining  z  bogles.  In  the  equation  for  the  K  values, 

Eqs.  k  12  and  6.  13,  z  and  z^  appear  in  both  equations.  Expand 
Eq.  6.  20  using  Eqs.  6.  12  and  6.  13.  All  the  terms  containing  z  can 
to  moved  to  one  side  of  the  equation  and  all  the  ‘erms  containing  z^  to 
the  other  side  of  the  equation. 


(6.  21) 


Ti  e  right-hand  side  of  this  equation  is  identical  with  the  left-hand 
side,  except  that  z^  appears  in  place  of  z  .  Let 


D(z)  -  $ 


+  . 


(6.  22) 


Equation  6.  21  is  simply 


D(za)  .  U(zb) 


(6.  23) 


The  value  of  Dfz^)  is  known.  If  the  function  D(z)  takes  on  each  of  its 
possible  values  exactly  twice,  then  there  would  always  be  a  solution 
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for  z  .  For  very  small  arguments  nearing  -oc,  the  normal  distribution 
function  approaches  zero.  Therefore,  for  very  large  negative  values 
of  z,  D  will  be  nearly  zero. 

D(-oc)  =  0  (6.  24) 

The  sign  of  the  derivative  of  D(z)  is  positive  for  z  values  less  than 
.  5zq  and  negative  for  z  values  greater  than  .  5z,q  . 

Sign  D’(z)  =  sign  (.  5z^  -  z)  (6„  25) 

This  means  that  D  takes  on  its  values  at  most  twice.  For  large  argu- 

z0 

ments  D(z)  approaches  1  -  e  . 

z0 

D(-hc)  =  1  -  e  U  (6.  26) 

If  the  mode  value,  zQ,  is  positive  the  D  function  takes 

on  negative  as  well  as  positive  values  to  the  right  of  .  5 zQ,  while  it  takes 

on  only  positive  values  to  the  left  of .  5Zq.  Therefore,  if  the  known 

value  z^,  which  is  below  the  mode,  is  also  below  half  the  mode,  then 

we  are  guaranteed  that  for  a  positive  zQ  a  solution  exists  for  Eq.  6.  23. 

If  Zq  is  negative,  the  range  of  values  to  the  right  of  the 

peak  of  D  is  less  than  the  range  of  values  to  the  left  of  the  peak. 

Therefore  if  z^  is  too  far  below  the  mode,  Zq,  there  will  be  no  value 

z  that  satisfies  Eq,  6.  23.  This  is  summarized  below, 
a 

z0 

Zq  <  0,  D(z^)  <  1  -  e  =£>  no  solution  for  z& 

(6.  27) 
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Given  the  information  in  Eq.  6.  16,  to  use  Procedure  I 
for  determining  the  fitting  ROC  character,  one  must  proceed  partly 
through  the  solution  before  knowing  whether  the  procedure  will  be 
successful  or  not.  The  solution  consists  essentially  of  Eq.  6. 19  for 
the  parameter  m  describing  the  strength  of  the  original  normal  ROC 
and  Eq.  6.  23  from  which  one  determines  the  upper  cut  value. 

First  Example:  Consider  the  ROC  character  of  the  conic  section 
which  is  Curve  1  of  Table  5.  4  and  Fig.  5.  13.  This  ROC  curve  and 
character  were  marched  to  an  extent  by  a  complete  normal  curve,  in 
Fig.  5.15.  The  equation  for  this  character  is 

,  -1.  5 

ir(z)  =  (705)*  D  (28  +  ez  +  e'z)  ,lzl  <  2.77 

(6.  28) 

Although  the  complete  ROC  character  is  known,  pick  out  simply  the 
lowest  value  of  the  range,  the  mode  value,  and  the  ratio  of  the  mode 
character  to  the  character  at  the  minimum  value. 

Given:  =  -2.  77  zQ  =  0  7t(Zq)/  =1.772 

(6.  29) 

Based  on  this  information  alone,  find  the  fitting  metastatic  ROC  char¬ 
acter  The  parameter  m  is  found  from  using  Eq.  6. 19.  Numerically 
this  results  in 

ir  =  (2.  77)/  V  2  In  1.772  =  2.  60;  m2  =  6. 76  (6.  30) 

(The  work  leading  to  Fig.  5.  15  utilized  a  normal  ROC  with  parameter 
d  =  2.  31  as  an  approximation. )  The  present  procedure  will  use  a  normal  ROC 
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with  strength  60  76,  but  will  not  utilize  the  entire  curve.  Since  the 
mode  is  zero,  the  equation  for  the  function  d  is  symmetric  and  the 
solution  for  the  upper  cut  value  is  immediate. 

B(za)  =  B(zb)  =?s>  za  =  +2.  77  (6.  31) 

All  that  remains  is  to  evaluate  the  K  parameter  using  Eqs.  6. 12  and  6. 13, 
and  to  writp  out  the  expressions  for  the  metastatic  normal  ROC  using 
Eqs.  6.14  and  6. 15,  This  results  in 


Kn  =  KgN  -.  400152 

(6.  32) 

X  =  [.991106  -  $(t  +  1.  3)]/.  400152 

Y  =  [.409046  -  $(t  -  1.3)]/.  400152 

(6.  33) 

A  character  with  known  ROC  curves  has  been  used  in 
order  that  we  might  evaluate  this  procedure  of  fitting  an  ROC  curve  by 
fitting  three  parameters  of  the  ROC  character.  The  true  ROC  curve 
for  this  case  has  been  computed  with  great  accuracy  on  the  IBM  7090 
computer  for  use  in  Chapter  V.  In  order  to  test  the  usefulness  of  the 
procedure  for  quick  hand  calculations,  all  of  the  parameters,  and  the 
ROC  curves  of  Eq.  6.  33,  were  calculated  using  a  handbook  of  normal 
tahles  and  an  ordinary  ten-inch  slide  rule.  The  result  of  the  ROC  curve 
evaluation  is  given  in  Table  6. 1,  along  with  the  true  value  of  the  ROC 
curve  as  obtained  by  its  computer. 


X 

.0125 

.0224 

.0496 

.0890 

.  1450 

.2200 

Y 

*MN 

.  160 

.249 

.  416 

1  ,560 

.685 

.782 

ytrue 

.  161 

.  251 

.418 

.  563 

.683 

.781 

Table  6. 1.  Comparison  of  true  and  approximate  ROC  curves 


The  combination  of  errors  in  the  procedure  and  in  the  slide  rule  technique 
resulted  in  a  discrepancy  that  can  not  be  observed  when  the  ROC  is 
plotted  in  the  one  percent  to  99  percent  range,  as  it  is  in  Fig.  6.  2. 

In  Fig.  6.  2  the  hyperbola  and  the  fitting  metanormal 
plot  on  top  of  each  other.  Also  on  that  graph  is  plotted  original  normal 
ROC  from  which  the  metastatic  normal  was  cut  out.  The  points  on  the 
original  normal  leading  to  the  metastatic  normal  are  indicated. 

To  summarize  this  example,  the  first  fitting  procedure 
has  been  used  on  a  shallow  symmetric  ROC  character.  The  resultant 
fit  utilized  only  forty  percent  of  an  original  normal  ROC  curve.  The 
resultant  lit  tothe  ROC  curve  was  excellent. 

Second  Example:  For  a  second  example  consider  the  nonsymmetric 
ROC  character  that  appeared  in  Fig.  5.  24,  and  whose  ROC  passes 
through  the  negative  diagonal  point  (.  3,  .7).  We  shall  use  only  the 
mode  value,  the  minimum  value,  and  the  ratio  of  the  ROC  character 
values  at  the  mode  and  the  minimum. 

Given:  Zq  =  -0.  2  z^  =  -2, 3  a(Zq)  /  iKz^)  -  7*  5  (6.  34) 
The  value  of  the  constant  m  is  found  by  Eq.  6. 19. 


m  =  (2. 1)/V  2  In  7.  5  =  1.04 


(6.  35) 


*v  ^ 
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In  order  to  utilize  the  D  function  of  Eq.  6.  22,  e  is  determined  and 
then  the  D  function  evaluated  at  the  lower  limit.  To  check  for  the  exis¬ 
tence  of  a  solution,  D(+x)  is  calculated  by  Eq.  6.  26.  The  results  are 

Z0 

e  =.81873  D(zb)  =  .  06345  D(+oc)  =.  18127  (6.36) 

The  value  .  06345  is  taken  on  only  once  by  D,  and  Procedure  I  yields 
no  solution. 

There  are  many  ways  to  explain  the  existence  of  this 
nonsolution.  The  most  apparent  is  that  for  this  value  of  mode,  the 
function  D  takes  on  more  values  to  the  left  of  its  peak  than  to  the  right 
of  its  peak. 

6.  3  Second  Procedure 

The  second  procedure  is  similar  to  the  first,  and  is 

utilized  when  the  first  fails.  The  first  procedure's  failure  meant  the 

D  function  took  on  more  values  to  the  left  of  its  peak  than  to  the  right 

of  its  peak.  The  second  procedure  also  uses  the  mode  z^,  and  hence 

the  same  function  D.  However,  if  the  known  value  is  z  ,  a  solution  is 

a 

guaranteed.  As  in  the  first  procedure  the  ratio  of  the  character  values 
at  the  mode  and  the  known  cut  are  given.  This  is  formalized  below. 

Known:  z&,  zQ>  that  zfe  <  zQ,  tt(z0)/  7r(za)  =  r&  (6.  37) 

As  in  the  first  procedure  this  amount  of  knowledge  is  sufficient  to 
determine  the  parameter  m.  The  expression  is 


d 


z  -  z_ 
a  0 


21nr 


(6.  38) 


The  second  part  of  the  solution  is  to  utilize  the  D  function  defined  by 


Eq.  6.  22  and  to  solve  Eq.  6.  23.  In  contrast  to  Procedure  I,  the  value 


of  the  upper  cut  z&  is  known,  and  Eq.  6.  23  is  utilized  to  determine  z^. 


Third  Example;  Let  us  return  to  the  example  for  which  the  first 


procedure  failed.  This  was  the  ROC  character  given  in  Fig.  5.  24 


with  the  negative  diagonal  ROC  point  (.  3, .  7).  The  information  neces¬ 


sary  for  the  second  procedure  is  the  mode  value  of  -0.  2  and  the  upper 


cut  value  of  +4.  6,  and  the  ratio  of  the  ROC  character  at  the  upper  cut 


and  the  mode.  The  values  are  given  in  Eq.  6.  39. 


Given:  zn  =  -0.  2  z  =  +4.  6 
0  a 


tt(z0)/  ?r(za)  =  288 


(6.  39) 


The  value  of  the  parameter  m,  this  time  using  Eq.  6.  38,  is 


m  *  (4.  8)/  /  2  In  288  =  1.  426 


(6.  40) 


In  this  second  procedure  m  is  1.  426,  whereas  it  was  1.  04  in  the  first 


unsuccessful  attempt.  First  step  numerical  values  are  shown  in  Eq.  6.  41. 


e  =  .  81873 


D(za)  =  .  18452 


D(-cc)  =  0 


(6.  41) 


D(-ac)  <  D(z  )  means  a  solution  to  Eq.  6.  23  exists.  The  numerical 
£1 


values  obtained  here,  and  the  trial-and-error  solution  for  z^  was 


carried  out  using  a  slide  rule.  The  solution  only  took  five  iterations, 
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and  the  accuracy  is  good  to  the  second  decimal  place  in  z^. 

\  =  -2.65 

J 

Kxt  -  1  -  ,  000023  -  .  193522  =  .  806455 
N 

Kcvt  =  1  -  .  004001  -  .  010983  =  .  985016 
SN 

X  =  [.999977  -  $(t  +  .713)]/.  806455 
Y  =  [.995999  -  $(t  -  .  713)]/ .  98  5016 


(6.  42) 

(6.  43) 

(6.  44) 


An  intuitive  feel  for  the  determination  of  the  metastatic 
ROC  curve  is  obtained  from  the  probabilities  of  occurrence  on  the 
original  axis.  This  is  sketched  in  Fig.  6.  3. 


193522  | 

. 806455 

. 000023 

-1.  578 

3.  365 

010983  i 

. 985016 

|  . 004001 

_ 

Z1 

Fig.  6.  3.  Sketch  of  probabilities  on  original  axis 


The  constant  ^  =  .  806455  means  that  this  amount  of  N-probability 
falls  between  the  two  cut  values  used  in  obtaining  the  metastatic  ROC. 
Almost  all  of  the  remaining  N-probability  fell  below  the  lower  cut,  with 
only  a  small  amount  being  chopoed  off  above  the  upper  cut.  The  fact 
that  the  mode  value  is  negative  means  the  value  of  KgN  is  bigger  than 
K^;  more  of  the  SN -probability  was  retained  between  the  two  cut 
values  than  was  N-probability.  Although  more  of  the  SN-probability 
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fell  below  the  lower  than  fell  above  the  upper  cut,  the  proportion  is 
nowhere  near  as  large  as  for  the  N  case. 

The  metastatic  normal  ROC  curve  of  Eq.  6.  44  and  the 
true  ROC  curve  cross  in  the  neighborhood  of  .  001  false  alarm  and  .  037 
detection  probability.  By  the  time  the  curves  get  into  the  ROC  region 
usually  displayed  on  norma? -normal  paper,  the  ROC  of  Eq.  6.  44  is 
about  .  1  in  d’  above  the  desired  ROC  curve. 

The  second  procedure  is  based  on  establishing  a  meta¬ 
static  ROC  character  that  has  the  same  maximum  z  value,  the  same 
mode  value,  and  the  same  ratio  of  ROC  character  at  mode  and  maximum 
z  values.  The  match  should  be  relatively  good  in  the  region  of  the  max¬ 
imum.  If  the  match  were  perfect  at  the  maximum;it  would  oe  perfect 
at  the  mode  value.  The  ROC  character  of  the  second  example  is  shown 
in  Eig,  6.  5  The  fit  falls  somewhat  below  the  true  value  at  the  maximum, 
z  =  4.  G.  This  is  relatively  unimportant  because  of  the  extremely 
small  value  of  the  character  here.  However,  it  is  reflected  in  a  cor¬ 
responding  discrepancy  in  the  neighborhood  of  the  mode,  whero  more 

probability  is  concentrated.  The  original  ROC  character  was  hyperbolic, 

**  1  5z 

and  the  limiting  slope  on  semi-log  paper  behaves  like  e  '  .  A 

parabola  cannot  match  such  a  straight  line  for  much  of  a  z-range. 
Therefore,  the  approximation  developed  in  Procedure  £1  is  first  below, 
then  above,  and  then  below  this  straight  line  section.  This  accounts 
for  the  ROC  curve  falling  above  the  desired  locus. 
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6.  4  Third  Procedure 

The  first  two  procedures  utilized  the  ratio  of  the  ROC 
character  at  the  mode  and  at  an  extreme  point.  The  matching  meta¬ 
static  normal  ROC  character  covered  the  same  relative  range  of  7r(z) 
values.  For  the  medium  probability  range  between  one  and  99  percent, 
the  behavior  of  the  ROC  character  when  its  magnitude  is  one  percent 
of  the  mode  magnitude  is  of  little  importance.  Another  characteristic 
of  any  ROC  character  which  appears  somewhat  parabolic  on  semi-log 
paper  is  the  "width”  of  the  hump.  Sharply  peaked  characters  corres¬ 
pond  to  poor  detection,  broad  characters  correspond  to  better  detection. 
Another  item  of  interest  about  the  ROC  character  are  the  points  on  the 
character  which  correspond  to  the  mode  of  the  z  distribution  under 
condition  N  and  condition  SN.  When  the  slope  of  the  derivative  of  the 
logarithm  of  the  character  is  +.  5;  the  derivative  of  the  N  density  function 
for  z  will  be  zero.  Correspondingly,  the  slope  5  at  a  somewhat 
larger  z  value  identifies  the  mode  of  the  SN  density  function.  The 
third  procedure  shall  assume  knowledge  of  one  of  the  extreme  values 
of  the  z-range,  and  the  position  of  the  two  modes  of  the  z  density 
functions.  If  these  two  modal  values  are  widely  separated,  then  the 
logarithm  of  n(z)  must  be  very  slowly  changing  slope,  and  be  very 
broad.  Conversely,  it  the  two  points  are  close  together,  the  ROC 
curve  corresponds  to  poor  detection.  Formalizing  what  is  known  for 


Procedure  m  we  write 
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Known:  z  or 
a 


dlnff(z) 
\  ■  dz 


=  rO.  5 


d  In  ?r(z) 


dz 


z=z_ 


z=z 


=  -0.  5 
(6.  45) 


z,  <  z  <  z ,  <  z 
d  c  d  a 


The  solution  for  this  procedure  is  fairly  straightforward. 
The  mode  of  the  matching  metastatic  normal  ROC  character  will  lie 
midway  between  the  two  points  with  slope  +  1/2  and  -1/2.  These  two 
points  also  determine  parameter  m. 

Zg  =  .  5(zc  +  z^)  ,  m  =  V  z^  -  z£  (6.  46) 

The  third  step  Is  the  same  as  the  third  step  in  the  first  two  procedures, 
namely,  to  solve  Eq.  6.  23  for  the  unknown  cut  value,  once  Zq  and  one 
cut  value  are  known. 

Fourth  Example:  As  an  example  of  this  third  procedure  consider  the 
same  character  for  which  the  first  procedure  failed,  and  for  which  the 
second  procedure  was  overly  generous.  From  the  graph  of  the  charac¬ 
ter  the  points  with  required  slope  were  estimated  to  be  -0.  6  for  the  N 
mode  and  +0. 7  for  the  SN  mode. 

Given:  z^  =  -2.  3  z^  =  -0.  6  z^=  +0. 7  (6.  47) 

In  rapid  order  solve  for  Zq  and  m. 

z0  „  ______ 

zQ  =  +0.05  e  =  05127  m  =  V  1.  3  =  1.  14 

Check  to  see  if  a  solution  for  the  D  equation  exists. 

* 


(6.  48) 


223 


D(z.  )  =  +.  06362  D(+oc)  =-.  05127  (6.  *9) 

c 

A  solution  of  the  D  equation  is  possible,  and  in  a  few  iterations  was 
determined  to  be 

z  =  2.  044  (6.  50) 

a 

The  next  steps  are  to  determine  the  K  constants  and  to 
obtain  the  detailed  equation  for  the  ROC  curves. 

Kn  =  1  -  .010170  -  .068112  =  .921718 

(6,  51) 

KgN  =  i  -  .  119000  -  .  004269  =  .  876731 
X  =  [.989830  -  $(t  +  .  57)]/.  921718 

(6.  52) 

Y  =  [.881000  -  $(t  -  .  57)]/.  876731 

A  sketch  very  much  like  Fig.  6.  3  has  been  drawn  for  this  case  and  is 
shown  in  Fig.  6.  6. 


I  1 

N  .068112  |  .921718  I  .010170 

_ -2.06 _ UL5 _ 

SN  .004269  |  .876731  !  .119000 

I  i 

Fig.  6.  6.  Sketch  of  probabilities  on  original  axis 


This  approximate  solution  leaves  less  of  the  SN-probability  in  the 
middle  than  N-probability,  and  that  far  more  of  the  high  z  values  are 
chopped  off  than  with  the  second  procedure. 
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The  ROC  curve  and  the  ROC  character  were  plotted 
in  Figs.  6.4  and  6.  5  respectively.  In  Fig.  6.  5  the  match  is  very  good 
all  along  the  ROC  character  from  the  minimum  point  up  to  the  neighbor¬ 
hood  of  the  mode.  The  corresponding  segment  of  the  ROC  curve  is  that 
dealing  with  the  high  probabilities.  However,  the  maximum  value  z  of 
the  metastatic  normal  is  only  a  little  over  two  compared  to  the  value 
of  4.  6  for  the  given  ROC.  On  the  ROC  curve,  as  a  point  proceeds  from 
high  probability  to  low  probability  this  metastatic  normal  drops  away 
from  the  desired  ROC  curve. 

6.  5  Summary 

The  purpose  of  these  procedures  is  to  deter  nine  the 
parameters  of  a  metastatic  normal  ROC  curve  whose  ROC  character 
matches  three  salient  distinguishing  items  of  a  given  character.  The 
corresponding  ROC  is  obtained  with  the  aid  of  normal  tables  and 


simple  calculation. 


CHAPTER  VII 


BINORMAL  ROC 


7.  1  General  Development 

The  binormal  ROC  curves  have  appeared  in  the 
literature  as  a  by-product  of  the  popular  use  of  normal  ROC  curves, 
and  use  of  normal-normal  graph  paper  for  plotting  empirical  ROC 
data.  Normal  ROC  curves  plot  as  straight  lines  on  normal- normal 
graph  paper,  with  a  slope  of  one,  that  is,  parallel  to  the  chance  diagonal. 
A  binormal  ROC  curve  plots  as  a  straight  line  on  normal-normal 
paper  but  with  slope  different  from  one. 

The  use  of  binormal  ROC  curves  adds  a  second 
parameter  to  the  class  of  ROC  curves  based  on  normal  distribution 
functions.  They  have  been  very  useful,  and  investigators  have 
learned  to  interpret  the  slope  as  indicative  of  additional  variance 
under  one  of  the  causes,  N  or  SN,  compared  to  the  other.  However, 
a  strict  interpretation  of  the  binormal  ROC  curve  is  as  a  nonregular 
ROC.  The  equations  for  the  relation  of  the  logarithm  of  the  likeli¬ 
hood  ratio  to  the  points  along  the  ROC  curve  will  be  developed  in 
Section  7.  2.  The  ROC  curve  can  be  made  regular  by  either  one  of 
two  (usually  slight)  modi' Nations.  The  first  of  these  is  treated  in 
Section  7.  3,  and  is  called  external  rectification.  External  rectification 
corresponds  to  adjoining  the  convex  closure  to  the  regular  part  of  the 
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binormal  ROC.  The  second  change,  made  in  Section  7.  4,  is  called 
internal  rectification.  Internal  rectification  corresponds  to  deriving 
an  ROC  based  on  the  likelihood  ratio  of  the  decision  axis  that  led  to 
the  original  nonregular  binormal  ROC  curve.  The  effects  of  these 
modifications  on  data  fitting  procedures,  and  upon  the  ROC  character, 
will  be  discussed  in  Section  7.  5. 

In  an  article  in  Science  (Ref.  27),  J.  A.  Swets  proposed 
the  ROC  curvt  as  a  basis  for  measuring  effectiveness  of  information 
retrieval  systems.  Figure  7  of  that  article  was  a  particular  type  of 
binormal  ROC  family;  it  is  redrawn  on  normal-normal  paper  in 
Fig.  7 .  1.  The  letter  E  is  Swets' index  for  effectiveness.  Although 
the  curve  with  effectiveness  E  =  0.  5  s  nearly  parallel  to  the  chance 
diagonal.  E  =  0.0,  as  the  effectiveness  increases  the  slope  is 
decreasing  slightly  with  each  successive  step.  These  curves  are 
binor mal  ROC  curves.  Figure  7.  2  is  Swets'  curve*  comparing 
empirical  data  to  the  binormal  ROC  curves  for  two  information 
retrieval  techniques.  Figure  7.  3  is  the  same  data  drawn  on  normal- 
normal  paper,  with  best-fitting  straight  lines  through  the  data.*  The 
values  for  s  marked  in  this  figure,  0.  79  and  0.  8o,  are  the  slopes  of 
the  best-fitting  straight  line  on  this  paper.  The  binormal  ROC  curves 
fit  the  data  very  much  better  than  the  best  normal  fit. 

1  By  permission  of  J.  A.  Swets.  These  appeared  in  an 
unpublished  work. 
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The  empirical  operating  characteristics  for  two  retrieval 
techniques  and  various  theoretical  operating  characteristics 
based  on  the  assumption  of  Gaussian  probability  distributions 
with  a  variance  proportional  to  the  mean,  specifically,  with  a 
ratio  of  mean  to  standard  deviation  of  4. 0. 


Fig.  7  2.  Swets'  first  curves 
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The  data  reproduced  on  probability  scales,  with  the  best- fitting 
theoretical  curves  assuming  Gaussian  probability  distributions. 
The  slopes  of  the  curves  are  the  best-fitting  slopes. 

Fig.  7.3.  Swots'  curves  on  normal- normal  paper 
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This  example  has  two  features  typical  of  applications 
of  the  binormal  ROC.  The  first  is  that  the  slope  is  less  than  one.  The 
second  is  that  only  a  portion  of  a  binormal  ROC  curve  has  been  utilized 
to  fit  the  data.  In  Fig.  7.  2  the  data  points  lie  near  or  to  the  left  of  the 
negative  diagonal. 

7.  2  Formal  Development 

One  form  of  the  equation  for  the  normal  ROC  curve  is 

X  $(-t)  Y  =  $(Vd  -  t)  (7.  1) 

The  function  $  (  )  is  the  normal  distribution  function.  When  a  normal 
ROC  curve  is  plotted  on  normal-normal  paper,  the  linear  horizontal 
coordinate  of  a  point  is  -t  when  the  linear  vertical  coordinate  is 
vTd-  t.  This  is  a  straight  line,  with  slope  one,  displaced  vertically 
above  the  chance  diagonal  by  a  distance  /d.  The  letter  s  is  often 
used  to  denote  the  slope  on  normal-normal  paper.  E  is  used  so  often 
in  probability  theory  for  expected  value,  that  the  letter  q  (for  quality) 
shall  be  used  as  the  index  of  effectiveness.  The  binormal  ROC  curves 
can  be  written  in  a  manner  analagous  to  the  normal  ROC  as 

X  •-  $(-t)  Y  =  $(s(q-t))  0  s  4  1  (7.2) 

The  geometric  interpretation  of  s  is  the  slope  of  the  ROC  curve  on 
normal- normal  graph  paper.  The  geometric  interpretation  of  the 
quality  index,  q,  is  the  distance  between  the  binormal  and  the  chance 
diagonal  at  the  fifty  percent  detection  point.  This  distance  may  be 


measured  either  horizontally  or  vertically.  Table  7.  1  gives  the 
separation  from  the  chance  diagonal,  the  d'  value  (  Ref.  10)  at 
several  locations  along  the  binormal  ROC  curve. 


Location 

Value  of  d’  =  $  "  *(Y)  -  $_1(X) 

- 1 

dY 

dX 

\  -  .50 

q 

.  5q2 
se  M 

Neg.  Diag. 

2sq/ ( 1+s) 

s 

rJ  2 

“  •  Q 

X  -  .  50 

sq 

se 

Table  7.  1.  Special  Values  of  d' 


Let  us  first  look  at  a  particular  binormal  graphically,  to  see  what  is 
objectionable  about  the  binormal  ROC.  Figure  7.  4  shows  a  binormal 
ROC  with  quality  1  and  the  rather  small  slope  of  1/2  drawn  on  normal- 
normal  paper.  In  this  form,  there  seems  to  be  very  little  that  is 
obviously  wrong  with  the  ROC  curve.  The  same  ROC  curve  is  displayed 
on  linear  coordinates  in  Fig.  7.  5(a).  On  this  scale  it  is  evident  that 
the  curve  is  an  S-shaped  curve,  and  not  convex.  From  the  point  (0,0) 
toward  the  ooint  (1, 1)  the  curve  convex  is  a  regular  ROC,  then  has  a 
nearly  linear  portion  where  the  slope  is  a  minimum,  and  then  the  slope 
increases  again,  in  fact  to  infinity,  as  the  ROC  goes  to  the  point  (1, 1). 

This  same  ROC  has  been  reproduced  in  Fig.  7.  5(b), 
together  with  the  chance  diagonal  and  one  other  straight  line.  The  point 
marked  C  in  Fig.  7.  5(b)  is  the  point  at  which  the  binormal  ROC 
curve  crosses  the  chance  diagonal.  From  this  point  on  to  the  right 


the  binormal  ROC  curve  falls  below  the  chance  diagonal.  Does  this 
always  happen?  The  binormal  ROC  curve  is  a  straight  line  on  normal- 
normal  paper,  and  not  parallel  to  the  chance  diagonal;  it  must  intersect 
the  chance  diagonal.  For  s  .  the  part  of  the  binormal  below  chance 
is  near  (1, 1);  for  s  ">  1  the  part  of  the  binormai  below  chance  is  near 
'0,  0).  The  point  on  the  ROC  curve  marked  with  an  arrow  and  zero 
indicates  the  point  of  minimum  slope  to  the  ROC  curve.  The  point  of 
minimum  slope  of  a  regular  ROC  curve  always  occurs  at  the  upper 
right-hand  corner  (1, 1).  Whenever  the  minimum  slope  occurs  before 
the  upper  right-hand  corner,  the  ROC  curve  must  be  nonregular.  The 
additional  line  drawn  on  this  figure  is  a  tangent  to  the  ROC  curve  that 
goes  through  the  upper  right-hand  corner.  The  point  of  tangency  is 
indicated  with  the  arrow  marked  T.  The  ROC  curve  formed  by 
piecing  together  that  part  of  the  binormal  ROC  curve  from  the  origin 
to  the  point  of  tangency  (marked  T)  and  the  straight  line  from  the  point 
T  to  the  upper  right-hand  corner,  is  a  regular  ROC  curve.  That 
part  of  the  binormal  ROC  that  is  used,  between  the  origin  and  the  point 
of  tangency,  is  called  "the  regular  part"  of  the  binormal  ROC.  The 
straight  line  between  the  point  of  tangency  and  the  upper  right-hand 
corner  is  called  the  "tangent  inferior.  ”  (The  tangent  line  to  any 
regular  ROC  curve  that  has  the  smallest  slope  of  any  tangent  is  the 
"tangent  inferior";  the  tangent  line  to  a  regular  ROC  curve  that  has 
maximum  slope  is  the  "tangent  superior.  ")  In  Fig.  7.  6  the  binormal 
ROC  curve  with  quality  q=2  and  slope  s=.  5  is  drawn  on  normal-normal 
paper  together  with  the  tangent  inferior.  The  point  of  tangency,  the 
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point  of  minimum  slope,  and  the  chance  diagonal  point  are  indicated. 

In  the  formal  development  that  follows,  the  restriction 
that  the  slope  falls  between  zero  and  one  is  made.  An  alternative 
restriction  would  be  that  the  slope  is  greater  than  one.  The  two 
cases  must  be  treated  separately,  but  the  results  are  similar  since 
the  two  cases  correspond  to  reflections  about  the  negative  diagonal. 

The  equation  for  the  binormal  ROC  curve  is 

X  =  5>(-t)  Y  =  S>  (s  (q  -  t)  )  0  <  s  <  1  (7.3) 

Since  X  and  Y  are  one  minus  the  distribution  functions  for  the 
dummy  random  variable  t,  the  probability  density  function  for  t  is 
obtained  by  differentiation. 

f(t|N)  =  -  0(-t)  (7.4) 

f(tiSN)  *  -  s0[s(q-t)]  (7.5) 

The  function  0(  )  is  the  normal  probability  density  function.  In  order 
to  determine  the  logarithm  of  the  likelihood  ratio  of  t,  divide  Eq.  7.  5 
by  Eq.  7.  4  and  take  logarithms. 

z(t)  =  In  s  -  .  5s2  (q-t)2  +  .  5(-tf  (7.  6) 

When  (7.  5)  is  expanded  in  terms  of  the  variable  t 

z(t)  =  In  s  +  .  5(1- s2  }  C  +  s£  qt  -  .  5s2  q*  (7.7) 
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Up  to  this  point  no  use  has  been  made  of  the  restriction 
that  s  falls  between  zero  and  one.  In  fact  all  of  the  equations  above 
hold  for  all  values  of  s  greater  than  zero,  including  one.  Were  s 
equal  to  one  the  quadratic  term  .  5(l-s2  )t2  would  drop  out  of  the 
Eq.  7.  7,  and  z  would  be  linear  with  t.  When  s  is  not  one,  z 
is  quadratic  in  t.  It  is  basically  this  quadratic  term  which  causes 
all  the  trouble  for  the  binormal  ROC;  this  quadratic  will  always 
appear  when  the  curve  is  binormal  and  not  strictly  normal. 

Use  of  the  assumption  that  s  falls  between  zero  and  one 
is  made  in  order  to  factor  out  the  term  1  -  s2  ,  complete  the  square, 
and  obtain  the  final  expression  for  z. 

z<t)  =  .5(1  -  s2  )(t  +  ~ +  (lr-  s  -  • 5  -pp  q2j  (7.8) 

The  minimum  value  of  the  quadratic  term  is  zero.  Therefore  the 
minimum  value  for  z  is  the  second  term  of  Eq.  7.  8.  Call  this  value 

V 

Zn  =  In  s-  .5  —2—  q2  (7.9) 

0  1  -  s2 

The  corresponding  value  for  the  dummy  variable  t  shall  be  called  t^. 


q 


(7.  10) 
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In  Figures  7.  5  and  7.  6  this  point  on  the  ROC  was  indicated  with  the 
index  zero.  The  coordinates  of  the  point  on  the  ROC  are  given  by 
Eq.  7.  11. 


xo '  *<-‘o>  q) 

(7.  11) 

Yq  =  *(s(q-t0)] 


From  this  point  of  the  binomal  ROC  curve  to  the  point  (1,1)  the  slope 

will  be  increasing.  The  slope  of  the  ROC  curve  is  the  likelihood  ratio, 

e  .  In  Eq.  7.  8,  as  the  quadratic  term  increases  either  because  t  is 

becoming  greater  than  tQ  or  because  t  is  becoming  less  than  t^, 

the  value  of  z  will  increase.  Since  the  slope  along  the  arc  from 

z0 

(X^,  Yq)  to  (1, 1)  is  increasing  it  is  greater  than  e  .  This  formal 
proof  of  the  concave  nature  of  the  binormal  ROC  curve  near  (1,1)  is 
summarized  in  .q.  7.  13. 


X0< 


=£> 


dY 


dY 


dX 


dX 


X 


(7.  13) 


0 


It  would  be  very  nice  to  be  able  to  solve  for  the  point 
of  tangency  of  the  tangent  inferior  as  simply  as  for  the  point  of  minimum 
slope.  In  order  to  do  this  consider  the  logarithm  of  the  slope  of  the 
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tangent  inferior. 


z  (tT ) 


1  -  Xr 


=  In 


$  (s(tT~q)  ) 


(7.  14) 


The  value  of  is  the  value  of  t  at  the  point  of  tangency.  An  explicit 
solution  by  equating  the  right-hand  sides  of  Eq.  7.  8  and  Eq.  7.  14  is 
not  possible.  The  graphic  solution  used  in  Fig.  7.  5  when  the  ROC 
is  drawn  on  linear  coordinate  paper  is  straight-forward  although  not 
extremely  accurate.  A  similar  procedure  can  be  used  on  normal- 
normal  paper  by  drawing  a  set  of  curves  on  that  paper  which  would 
be  straight  lines  from  the  point  (1, 1)  on  linear  paper.  The  point  of 
tangency  between  the  tangent  inferior  and  the  binormal  can  again  be 
judged  rapidly  but  not  extremely  accurately  by  this  graphical  method. 
The  only  guaranteed  accurate  method  known  to  the  writer  is  an  iterative 
solution  for  t^,  by  tables  and  hand  computation,  or  by  computer.  The 
values  obtained  in  the  present  paper  were  obtained  using  an  IBM  7090 
computer. 

Before  proceeding  to  various  methods  of  rectifying  this 
irregular  ROC  curve,  the  ROC  character  for  the  regular  part  of  the  arc 
is  obtained.  The  equations  also  hold  for  the  irregular  part  of  the  arc, 
but  the  intended  application  is  to  the  regular  part  from  the  origin  to 
the  point  of  tangency  with  the  tangent  inferior. 


Equation  7.  8  is  simplified  by  using  Eqs.  7.  9  and  7.  10. 
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z  =  .  5(1  -  s2  )  (t  -  tQ)2  +  zQ 
Solve  Eq.  7.  15  for  t 


t 


t 


0 


(z 


v] 


(7.15) 


(7.16) 


The  aiuation  for  the  HOC  curve,  in  terms  of  the  logarithm  of  the  likeli¬ 
hood  ratio,  is  therefore 


The  Jacobian  of  transformation  from  t  to  z  is  the  derivative  of  t 
with  respect  to  z. 


=  [2(1  -  s2  )  (z  -  zQ)J"-5  (7.18) 

The  density  function  for  z  under  condition  N  is  obtained  by  the  substi¬ 
tution  of  Eq.  7.  16  into  Eq.  7.  4,  and  then  multiplying  by  Eq.  7.  18 


dt_ 

dz 
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f(ziN)  = 


\Z2tt  \/2(l-s2  ) 


(7.  19) 


v  5 

tz  -  zQ)  e 


-[K 


2  2  s/2  .  , - 

+  — —  *0 


'1 

V  1“S 


l-s~ 


<*'Z0>] 


It  will  be  convenient  to  allow  the  term  t^  to  remain  for  several  more 

steps.  The  ROC  character  is  obtained  from  the  N  density  function 

5z 

by  multiplying  by  e'  The  form  used  is 


77(z) 


.5(z-zq)  .5z 

e  e 


f(z  i  N) 


(7.  20) 


Equation  7.  9  yields 


e 


(7.21) 


This  leads  to  the  following  form  of  the  ROC  character 


5  . 25qtQ 

e 


-.5 


(z-z0) 


(7.22) 


The  final  step  is  to  eliminate  the  constant  t^  by  using  its  definition, 
Eq.  7.  10.  The  final  form  of  the  ROC  character  is 
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Equation  7.  23  is  so  complicated  with  clusters  of 
coefficients  that  it  is  difficult  to  envision  its  behavior.  It  is  rewritten 
to  focus  on  those  places  where  ..he  variable  z-z^  appears. 

ti(z)  =  A(z-z0f-b  eByfi~*0  "C(z'z0)  (7.24) 

If  the  quality  is  positive,  as  we  may  assume  it  to  be  in  most  interesting 
cases,  the  coefficients  A,  B  and  C  will  be  positive.  For  a  very  large 
magnitude  z  the  final  exponential  term  will  dominate,  and  the  ROC 
curve  will  behave  similarly  to  one  with  an  exponential  character,  a  pure 
power  ROC.  If  the  constant  B  were  zero  (it  is  not),  then  the  character 
would  be  that  of  a  Pearson  III  with  p  ~  -■  5.  Such  a  Pearso*.  charac¬ 
ter  is  monotone  decreasing  in  z.  The  term  By/z-z^  tends  to  make 
the  character  increase  as  z  increases..  This  means  that  the  charac¬ 
ter  may  exhibit  a  mode,  or  may  be  monotone  decreasing  in  z,  depending 
on  the  specific  values  of  the  parameters. 

In  this  section  the  ROC  character  for  the  regular  part 
of  the  binormal  ROC  curve,  Eq.  7.23  was  obtained.  The  equation 
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for  the  ROC  curve  it s ell  in  terms  of  the  slope  and  quality  parameters, 
and  the  logarithm  of  the  likelihood  ratio  z  was  also  determined.  This 
is  Eq.  7.  17. 

7.  3  External  Rectification 

The  process  of  external  rectification  consists  of  replacing 
the  irregular  part  of  the  ROC  curve  with  the  tangent  inferior.  The  corres¬ 
ponding  action  on  the  ROC  character  is  to  limit  the  applicability  of 
Eq.  7.  23  to  those  z  which  fall  above  zT  ,  (the  logarithm  of  the 
slope  of  the  tangent  inferior  )  and  to  add  a  jump  in  the  integrated  ROC 
character  at  of  appropriate  magnitude. 

No  analytic  solution  for  the  tangent  inferior  has  been 
found.  Therefore  a  computer  program  was  written  that  did  the  following 
search.  The  binormal  ROC  curve  was  traced  starting  at  the  origin.  At 
each  point  along  the  ROC  curve,  the  value  of  z  was  compared  with  the 
(logarithm  of)  the  slope  from  the  point  on  the  ROC  to  the  point  (1,1). 

Near  the  origin  the  line  connecting  the  ROC  point  with  the  point  (1, 1) 
is  a  secant,  interior  to  the  ROC.  As  soon  as  the  two  slopes  agree,  the 
secant  has  become  a  tangent,  and  the  program  halts.  This  is  a  straight¬ 
forward,  albeit  brute  force,  method  for  determining  the  tangent  inferior 
and  2’lrp.  This  program  is  listed  in  Appendix  F. 

Figure  7.  7  presents  five  ROC  characters  which  were 
plotted  as  part  of  one  run  of  this  program.  For  this  run,  the  slope 
parameter  s  was  fixed  at  0.  8.  The  qualities  examined  were  q  =  1,2, 3, 4, 5. 
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In  Fig.  7.7  the  points  on  the  ROC  character  with  positive  z  marked 
with  a  large  open  dot  indicate  the  point  at  which  the  ROC  curve  passed 
through  .  01  false  alarm  probability.  The  points  marked  with  the 
solid  dot  indicate  the  points  at  which  the  ROC  curve  passed  through 
the  .  99  detection  probability.  When  the  quality  parameter  is  one, 
the  ROC  character  is  strictly  monotone  decreasing.  In  the  neigh¬ 
borhood  of  Zq  the  ROC  character  is  very  steep.  The  other  four 
ROC  characters  present  a  more  normal  appearance.  Each  is 
terminated  at  with  an  cu  ^  term.  For  q  =  2  and  greater, 

z^  was  sufficiently  to  the  right  of  to  prevent  the  appearance 

-.  5 

of  a  sharp  rise  in  the  ROC  character  due  to  the  term  ( z-z^ )  ’  . 

The  corresponding  ROC  curves  are  plotted  on  normal- 
normal  paper  in  Fig.  7.  8.  The  effect  of  the  external  rectification  is 
evident  only  in  the  q  =  1  curve.  As  was  seen  on  the  ROC  character, 
all  of  the  other  curves  contain  the  one  percent  to  99  percent  region  in 
their  regular  portion.  Although  it  is  important  in  theory  to  make  sure 
that  the  ROC  curves  are  regular,  it  may  very  well  turn  out  in  practice 
that  this  careful  consideration  has  been  unnecessary.  In  order  to 
exhibit  binormal  ROC  curves  which  evidenced  need  for  rectification  on 
the  usual  plot  on  normal-normal  paper,  a  number  of  cases  with  slope 
equal  to  one-half  were  considered.  These  are  presented  in  Fig.  7.  9. 
The  points  where  these  would  cross  the  chance  diagonal  are  marked 
wiLh  C  and  the  points  where  these  would  take  on  their  minimum  slope 


248 


e  ^  are  marked  with  0.  The  point  at  which  the  tangent  inferior  is 
adjoined  to  the  curve  to  make  it  regular  is  marked  with  T. 

The  curves  shown  in  Fig.  7.  9  are  regular  ROC  curves, 
derived  from  binormal  ROC  curves  by  external  rectification.  The  cor¬ 
responding  integrated  cnaracter  has  a  jump  at  the  value  z^,  and 
is  differentiable  from  z ^  to  infinity. 

7.  4  Internal  Rectification 

The  binormal  ROC  curves  are  irregular  because  they 
are  based  on  a  decision  axis,  t  ,  which  is  not  monotone  with  its  own 
likelihood  ratio.  External  rectification  removed  part  of  the  axis  from 
consideration  in  order  to  maintain  a  monotone  relationship  between  the. 
remaining  range  of  t  and  the  likelihood  ratio.  A  decision  mechanism 
that  allows  access  to  the  t  axis  can  undergo  a  superior  rectification 
to  the  simple  external  rectification.  Without  access  to  the  decision 
axis  there  is  no  other  choice  but  to  stay  with  external  rectification. 

Consider  a  decision  device  with  a  decision  axis,  t, 
which  led  to  a  specific  binormal  ROC  curve  with  quality  q  and  slope 
s  between  zero  and  one.  The  probability  density  functions  for  this 
decision  axis  are  both  normal,  and  are  given  in  Eq.  7.  4  and  7.  5. 

Since  the  logarithm  of  the  likelihood  ratio  is  quadratic  in  t.  then 
the  response  "A”  corresponding  to  z  being  greater  than  some 
ievel  will  be  equivalent  to  a  two-tailed  test  on  the  t  axis. 


249 


The  probability  density  function  for  z  under  both 

of  the  causes,  N  and  SN,  is  based  on  the  sum  of  the  probability 

densities  for  each  of  two  appropriate  t  values  (the  solution  of  7.  15). 

The  ROC  character  may  be  determined  as  the  probability  density 

i  5  z« 

for  z  under  one  of  the  causes,  multiplied  by  either  e  '  '. 

Another  approach  is  to  find  the  root  likelihood  product,  and  to 
multiply  this  by  the  Jacobian  of  transformation,  dt/ dz. 

The  root  likelihood  product,  using  Eq.  7.  3  and  7.  4, 
and  taking  the  square  root,  is 


[ f(t  | N)  f (t |  SN)]  =  [  gf -]'5  e 


+  S2  (q-t)2  ] 


(7.25) 


In  tiie  previous  section  the  relation  between  z  and  t  was  derived 


t 


(z-2o) 


.  5 


(7.26) 


This  differs  from  the  previous  equation,  Eq.  7.  16,  in  the  range  of 
application.  Previously  t  had  been  limited  to  those  values  t  greater 
than  tg,  in  fact,  those  values  of  t  greater  than  t^.  However, 
direct  use  of  the  likelihood  ratio  allows  use  of  the  entire  t  axis.  A 
transformation  of  variables  effects  the  probability  density  functions 
by  the  ab^lute  value  of  the  Jacobian  of  transformation  and  not  the 
signed  value.  Therefore,  Eq.  7  18  holds  independently  of  whether  the 
plus  or  the  minus  sign  in  Eq.  7.  26  is  used. 
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|-||  =  [2(1-s2)(z-z0)]'-5  (7-18) 

The  only  remaining  step  is  to  substitute  t  into  the  expression  in  the 
exponent  of  Eq.  7.  25  and  do  some  careful  algebra.  Expand  the 
exponent  of  Eq.  7.  25  and  use  Eq.  7.26  to  obtain 


+  s“  (q-tf 


1+s2  1 

1-s2  J 


(z-zQ)  + 


4n£s2 


(1-s"  ) 


2  a.  5 


(z-z 


v-5 

O' 


(7.27) 


s2  q2  (1+s2  ) 
(1-s2  f 


The  ROC  character  is  obtained  by  adding  together  the  two  values  for 
the  root  likelihood  product  that  correspond  to  the  same  value  of  z, 
and  multiply  the  result  by  Eq.  7.  18.  Two  of  the  factors  in  the 
exponent  ,  Eq.  7.  27,  are  common.  That  is,  two  of  the  terms  in 
Eq.  7. 27  are  independent  of  whether  the  plus  sign  or  the  minus 
sign  in  Eq.  7.26  is  used.  The  Jacobian  of  transformation  is  also 
independent  of  which  sign  has  been  used.  Therefore,  the  only  effect 
of  the  summation  is  on  the  middle  term  in  Eq.  7.  27.  Since  this 
middle  term  appears  in  an  exponent,  the  adding  of  the  two  terms 
will  yield  a  hyperbolic  cosine  term.  The  result  of  the  transformation 


is  shown  below. 
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2(l-s2  ) 

The  main  change  between  Eq.  7.  28  and  Eq.  7.  23 


appears  to  be  that  the  sing.e  exponential  term  involving  a  square  root  of 

z  has  become  a  hyperbolic  cosine  term  involving  the  square  root  of  z. 

Another  basic  difference  which  is  not  so  evident  is  that  Eq.  7.  28  is 

valid  for  z  right  down  to  z^,  whereas  Eq.  7.  23  is  valid  only  for  z 

as  iov;  as  z^,,  which  is  greater  than  z^.  This  means  that  the  minimum 

slope  of  the  internally  rectified  binormal  ROC  curve  will  be  the  same 

as  the  minimum  slope  of  the  original  irregular  binormal  ROC  curve. 

The  integrated  ROC  character  for  the  internally  rectified  case  contains 

no  jump,  that  is.  there  is  no  w  term  at  the  lower  end  of  the  ROC 

character.  However,  there  is  an  action  in  the  ROC  character  which 

behaves  very  similar  to  this  "delta-f  unction.  "  Since  z  is  no  longer 

bounded  away  from  z^ ,  in  the  immediate  neighborhood  of  z^the 

,  5 

ROC  character  will  be  dominated  by  the  term  u-Zq)  .  As  z  gets 
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smaller  and  smaller  the  character  will  suddenly  become  strongly 
influenced  by  this  term  ana  will  approach  infinity. 


(7.  29) 

An  example  of  this  behavior  in  the  neighborhood  of 


Zq  is  shown  in  Fig.  7.  10,  a  plot  of  the  logarithm  of  the  ROC  charac¬ 
ter  for  the  case  q  =  1  ,  s  =  .  8  .  The  ROC  character  for  the  exter¬ 
nally  rectified  case  was  shown  in  Fig.  7.  7.  The  characters  for  both 
the  internal  and  external  rectification  appear  very  similar.  Tne  only 
difference  is  that  the  ROC  character  for  the  internally  rectified  case 
is  continuous  and  has  no  u)^  term  adjoined  to  it.  However,  the 
appearance  is  very  much  the  same  for  both  cases. 

A  detailea  comparison  of  the  behavior  near  Zq  for 
internally  and  externally  rectuied  ROC  characters  is  shown  in 
Fig.  7.  11.  The  case  considered  is  q  =  2  ,  s  =  .8  .  The  value 
of  z  is  about  .01  greater  than  the  value  Zq  .  For  the  internally 
rectified  case  the  ROC  character  is  monotone  decreasing  as  it 
approaches  z ^  from  higher  values,  the  then  u;^(z  )  is  abruptly 
adjoined.  For  the  internally  rectified  case  the  ROC  character  de¬ 
creases,  passes  through  a  minimum,  and  rises  abruptly  to  infinity 
as  it  approaches  Zq  from  above. 


Fig.  7.12.  Binormal  ROC  curves 
internally  rectified 


I 

i 
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The  computer  program  that  accomplishes  internal 
rectification  is  also  in  Appendix  F.  The  case  s  =  .5  that 
was  analyzed  for  external  rectification  was  also  analyzed  for 
internal  rectified  ROC  curves.  Figure  7.  12  snows  the  results  of 
internal  rectification  when  the  original  quality  was  q  =  1,  2,  3,  and  4. 
These  may  be  compared  with  the  externally  rectified  solution  given 
in  Fig.  7.  9.  Curves  for  internal  rectification  are  smoother,  and 
somewhat  better  than  for  the  externally  rectified  case.  The  appearance 
of  the  q  =  4  curve  is  virtually  unchanged,  however,  the  additional 
increases  being  in  the  third  or  fourth  decimal  place. 

7.  5  Discussion 

This  chapter  considered  binormal  ROC  curves,  which 
plot  as  straight  lines  at  slope  less  than  one  on  normal- normal  paper. 
These  are  irregular  ROC  curves,  but  we  can  associate  with  any 
binormai  curve  a  regular  ROC  curve  by  either  external  or  internal 
rectification.  Such  rectification  changes  the  shape  of  the  curve,  and 
changes  the  ROC  character. 

A  very  important  question  is  the  actual  magnitude  of 
me  changes  in  the  ROC  curve,  especially  if  one  is  interested  in  a 
amited  region  of  the  ROC  curve.  In  the  first  section,  ROC  curves 
were  taken  from  some  work  of  J.  A.  Swets.  For  the  family  of  curves 
used,  the  slope  is  related  to  the  quality  by  the  equation 

1 

1  +  .  2'5q 


s 


(7.  30) 
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It  can  be  shown  that  all  of  the  ROC  curves  in  this  family  pass  through 
a  common  point  on  the  chance  diagonal.  This  is  the  point  (.  999968, 

.  999968).  The  data  (Fig. 7.  2,  7.  3  )  fell  in  the  medium  percentage 
levels,  in  fact,  to  the  left  of  the  negative  diagonal,  and  were  quite 
distant  from  probabilities  such  as  .  999968.  The  program  developed 
for  this  work  internally  rectified  six  curves  from  this  family.  The 
result  is  shown  on  extended  normal-normal  paper  in  Fig.  7.  13. 

The  resulting  rectified  ROC  curves  do  not  cross  the  chance  diagonal, 
and  do  not  cross  each  other.  These  curves  and  their  corrections  on 
the  usual  range  of  normal-normal  paper  is  Fig.  7.  14.  In  the  region 
above  .  97  detection  the  rectified  ROC  curve  is  slightly  better  than 
the  original  binormal  curve.  One  can  certainly  conclude  that  Swets 
is  in  no  trouble  from  having  chosen  to  operate  with  a  nonregular 
ROC  curve.  The  rectification  is  formally  called  for  if  a  rigorous 
treatment  is  desired  for  the  ROC  curves.  As  a  descriptive  model 
of  empirical  data,  a  way  of  fitting  and  describing  data,  the  curves 
are  perfectly  valid.  In  fact,  one  may  undoubtedly  manipulate  the 
model  mathematically  so  long  as  the  operations  stay  concentrated 
on  the  region  far  from 

3inormal  ROC  curves  have  arisen  in  detection  problems 
with  slopes  as  drastic  as  s  =  .  5.  When  such  curves  are  to  be  used 
in  a  region  anywhere  close  to  the  chance  diagonal,  one  or  the  other 
ivpe  o't  rectification  must  be  considered  to  render  the  ROC  curves 
regular.  The  final  figure  of  this  section.  Fig.  7.  15,  shows  two 


Fig.  7.15.  Binormal  ROC,  s  =  .5  showing 
two  types  of  rectification 
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binormal  ROC  curves  with  slope  equal  to  .  5.  Both  the  internal 
and  external  rectification  have  been  shown  on  the  same  figure. 

If  one  is  dealing  with  slopes  as  drastic  as  this,  he  may  very  well 
wish  to  incorporate  rectification  into  his  model. 


CHAPTER  Vm 


/ 


ROC  FOR  MULTIPLE  OBSERVATIONS 

8. 1  Convolution  Theorem 

The  distribution  for  the  sum  of  a  number  of  independent 
random  variables  can  be  determined  by  convolution  of  the  distributions 
for  the  individual  random  variables.  In  this  section,  the  convolution 
theorem  for  the  ROC  characters  is  developed  for  characters  without  jump 
functions.  The  convolution  concept  is  important  in  detection  situations 
in  which  the  observation  upon  which  a  single  decision  is  based  consists 
of  several  independent  parts.  The  logarithm  of  the  likelihood  ratio  for 
such  an  observation  is  the  sum  of  the  logarithm  of  the  likelihood  ratios 
of  the  independent  parts.  Let  z.  denote  the  logarithm  of  the  likelihood 
ratio  for  the  i-th  part  of  the  observation,  and  jt^z.)  denote  its  ROC 
character.  Let  z  be  the  logarithm  of  the  likelihood  ratio  for  the  com¬ 
plete  observation. 


n 


2  =  V 


Cl" 


(8.1) 


j 

1 


Theorem:  If  ir.(z.)  are  the  ROC  characters  for  statistically  inde 


pendent  observations,  then 


where  toe  star  indicates  convolution 
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Proof:  The  proof  is  inductive,  and  depends  on  the  convolution  theorem 
for  probability  density  functions. 


Initial  Step:  n  =  2  z  =  z^  + 


f(z;SN)  =  J'f^Zj 


SN)  f2(z2 


SN)  dz 


.  5Zj  .  5(z  -  z^) 

=  J  e  1  ff1(z1)  e  1  7r2(z  -  Zj)  dzx 


=  e'  5z  /  ^(Zj)  77 2 (z  -  Zj)  dzx 


77 (z)  =  e~*  5z  f(z  !  SN) 

.  '  .  77 (z)  =  J  771(zi)  77 2 (2  -  Z )  dz 


Inductive  Step:  Assume  that  the  theorem  is  true  for 
n  =  r  2. 


Let 


z  - 


r+1 

S' 

Lu 

i=l 


z. 

1 


Z  = 


r 

V 

u 

i=l 


z 


i 


=  Z  +  z  * 
r  +  1 


z 
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Proof  (Cont. ) 

by  the  initial  step,  and  the  inductive  hypothesis 

7r(z)  =  [  n  (z- )  *  v .(z.j  *. . .  ir  (z  )]  *  n  -  (z  1 ) 

L  i  1  2  2  r  r'J  r  +  1  r  +  r 

Convolution  is  associative,  so  we  may  omit  the  [  ]  . 

Q.  E.  D. 


The  convolution  of  the  classical  distributions  are  well 
known.  Brief  comments  about  the  normal,  the  rectangular,  and  the 
exponential,  in  order  to  call  attention  to  certain  specific  ideas. 

The  convolution  of  a  finite  number  of  normal  density 
functions  is  again  normal,  with  the  mean  being  the  sum  of  the  means 
and  the  variance,  the  sum  of  the  variances.  The  normal  character  is 
proportional  to  a  zero  mean  normal  density,  with  variance  d.  There¬ 
fore,  the  convolution  of  a  number  of  normal  characters  will  yield  a 
normal  character,  symmetric  about  zero,  and  with  detection  index  d 
being  the  sum  of  the  detection  indices  for  the  individual  z .  . 


The  rectangular  ROC  character  is 

i 


»,<*!>  ’  v1;rR  ,  |  |  <  )Ji(l  +  R) 


(8.3) 


(8.4) 
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The  convolution  of  two  identical  such  rectangular  characters  is  given 
in  Eq.  8.5. 

i 

■  i 

tf(z  =  zt  +  z0)  =  —  +~—  I  z  -  2  In  (1  +  R)  | 

1  2  4R2 

|z|<  2  In  (1  +  R)  (8.  5) 

This  is  a  triangular  ROC  character.  The  convolution  of  n  identical 
rectangular  characters  will  be  expressed  in  n-th  degree  polynomials 
in  z.  The  expression  will  consist  of  n  +  1  sections  along  the  z 
axis,  and  become  very  cumbersome.  The  result  has  a  central  tendency 
to  the  normal  character.  It  would  be  useful  to  have  a  measure  of  the 
normal  parameter,  d,  which  could  be  used  as  the  number  of  individual 
rectangular  characters  is  large.  The  usual  procedure  is  to  equate 
variances.  In  order  to  do  this,  we  would  have  to  re-normalize  the 
ROC  character  to  have  unit  area.  An  equivalent  calculation  for  sym¬ 
metric  characters  is  to  define  a  quantity  such  as 

$  -  J  z2  y(z)  dz 

/  7r(z)dz 

For  the  rectangular  character  of  Eq.  8.  4 

\  =  |  ^ln  (1  +  Ryj  ? 

Eq.  8.  6  evaluated  for  the  character  resulting  from  n  convolutions  of 
similar  ROC  characters  will  be  n  times  as  large  as  for  the  single 


(8.6) 


(8.  7) 
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ROC  character. 

K =  1 10-1 +r>)2  (8.8) 

The  gamma,  Pearson  m,  and  chi-square  distributions 
all  correspond  to  the  same  ROC  character.  It  is  well  known  that  the 
sum  of  chi-square  random  variables  has  a  chi- square  distribution, 
and  similarly  for  the  other  distributions.  These,  in  turn,  are  strongly 
related  to  the  simple  exponential.  Consider  the  simple  exponential 
ROC  character  given  in  Eq.  8.9. 


A  *  Z.  - 

A  +  1  i  0 

7T . (z . )  =  -A-y  e  A  "  *  2  z.  >  zn  =  -In  A  (8.9) 

i  i  A  -  1  x  u 

If  n  such  characters,  ail  with  the  same  index  A,  are  convolved,  the 
result  is  the  Pearson  HI  character  with  the  same  index. 


This  result  does  demand  that  the  indices  A  be  identical  for  all  of  the 
individual  characters. 


8.  2  Discrete  Characters 

The  proof  used  in  8, 1  for  the  convolution  of  ROC  char¬ 
acters  was  given  in  terms  of  probability  density  function.  A  similar 


result  holds  for  discrete  ROC  characters.  It  will  be  assumed  without 


proof.  The  simplest  discrete  character  is  the  Luce  character  a  ah 
only  two  points.  The  ROC  curve,  such  as  shown  in  Fig.  8. 1(a),  con¬ 
sists  of  two  straight  lines  meeting  at  the  point  (x^,  y^),  &tso  marked  as 
the  vertex  q.  In  the  general  binomial  ROC  curve,  let  the  vertices 

be  marked  V  ,  indicating  that  n  observations  were  made,  and  k 
n,  k 

of  these  or  fewer  were  indicative  that  the  condition  was  N.  As  with 
any  polygon,  it  is  easiest  to  write  out.  the  total  polygon  by  naming  the 
vertices.  The  Luce  ROC  curve  is  specified  by 

(0,0)  VljQ  =  (xr  yt)  V11  =  (1,1)  (8.11) 

If  two  independent  observations  are  made,  and  each  has 
the  same  Luce  ROC  character,  then  three  likelihood  ratios  are  possible. 
Because  of  the  statistical  independence  and  similarity  of  the  observa¬ 
tion,  order  is  unimportant.  The  ROC  curve  resulting  from  Fig.  8.1(a) 
is  shown  in  Fig.  8. 1(b).  This  is  a  Green  double  threshold  ROC.  What 
has  previously  been  referred  to  as  the  Green  ROC  is  the  limiting  form 
of  the  Green  double  threshold  ROC,  where  the  vertices  V 2  ^  and 
V2  ^  lie  arbitrarily  close  to  the  edges  of  the  graph.  The  vertex  V 2  Q 
corresponds  to  the  response  "A"  made  whenever  neither  observation 
was  indicative  of  condition  N.  The  vertex  V„  .  corresponds  to  the 
response  "A”  whenever  one  or  fewer  observations  were  indicative  of 
condition  N.  The  description  of  this  ROC  by  its  vertices,  together  with 
the  relation  of  these  vertices  to  the  Luce  vertex  is  given  in  Eq.  8. 12. 
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(0,0)  V2  0  -  (x^.y^)  V2  ,  =  (2xx  -  Xl2,  2yt  -  y^) 

V2 >2  =  d,l)  (8.12) 

In  the  particular  case  graphed  in  Fig.  8. 1,  the  Luce  vertex  V1  n  falls 
on  the  Green  double  threshold  ROC  curve.  It  falls  exactly  in  the 
middle  of  the  line  segment  between  the  two  middle  vertices.  This 
will  always  happen.  The  notation  that  half  of  one  vertex  plus  half  of 
another  vertex  means  the  point  lying  halfway  between  the  two  vertices 
indicates  both  the  geometric  relation,  and  the  algebra  that  must  be 
performed  on  the  coordinates  to  obtain  this  geometric  relation. 
Equations  8. 12  for  the  Green  double  threshold  ROC  and  8. 11  for  the 
Luce  ROC  can  be  combined  to  obtain: 


2,0 


<*!■  V  * 


(8.13) 


If  three  independent  observations  are  taken,  and  each 
has  the  same  Luce  ROC,  the  resultant  ROC  will  have  three  nonchance 
vertices.  These  are  listed  below 


(0,0) 


v3,3  *  <ri> 


(8.14) 


and  the  equations  for  the  nonchance  vertices  in  terms  of  the  Luce 


vertex  are 
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V3.0  =  lxl’-  yi’> 


V3>j  =  (3xj2  -  2xj3,  3yj2  -  2yj5) 


2  "  (3x^  -  3Xj  t  Xj  ,  3y  ^  -  3  y  ^  ) 


(8.15) 


Examination  of  Eq.  8. 15  and  the  equations  for  the  Green  double  thres¬ 
hold  ROC  vertices,  Eq.  8.12,  reveals  that  the  Green  double  threshold 
vertices  lie  along  the  segments  of  the  three  observation  ROC. 


3  V3,G  +  3  V3,l  “  (X1*’  yl2)  ~  V2, 


(8.16) 


3  V3. 1  +  3  V3,2  "  (2xl  "  xi  >  ~  yx2)  -  v3  0  (8-17> 


It  appears  that  each  successive  ROC  curve,  due  to  one  additional  obser¬ 
vation.  is  better  than  the  previous  ROC,  but  not  strictly  better.  Each 
vertex  of  the  prior  ROC  touches  a  segment  on  the  new  ROC.  This  is 
indeed  the  case.  The  theorem  will  be  stated  in  terms  of  the  vertex 
notation;  however,,  the  proof  itself  will  deal  with  the  coordinates  of 
the  vertex.  The  meaning  of  the  vertex  notation  is  written  out  explicitly 

Vn.  k  =  (Xn,  V  Yn,  k>  •*-»  jn  observaUon'  -  k  "3"  j  <8- 18) 


Theorem: 


n  -  l?k 


0  <  k  <  n 


i 
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Proof: 

Assume  that  n  observations  have  been  taken,  and  thai  each 
has  a  Luce  ROC  character  associated  with  the  Luce  point 
(Xj,  y^)  .  The  k-th  ROC  point  corresponds  to  responding  "A" 
whenever  there  have  been  k  or  less  observations  indicative 
of  condition  N  .  The  probability  of  a  false  alarm  is  there¬ 
fore  given  by  a  partial  sum  over  the  binomial  distribution 


(1-  Xj)j 


(8.19) 


The  probability  of  a  detection  is  given  by  a  similar  sum 


l  c^u-y^y-* 

j=0  3  11 


(8.20) 


The  only  difference  between  Eqs.  8. 19  and  8.  20  is  the  use  of 
for  the  condition  N  and  y^  for  the  condition  SN  .  The 
following  mar  ipulations  will  deal  with  the  N  condition.  Simi¬ 
lar  results  for  the  SN  condition  may  be  obtained  by  inter¬ 
changing  Xj  and  y ^  .  Most  of  the  specific  manipulations 
have  been  placed  in  Appendix  G  and  only  the  key  results  dis¬ 
played  here.  Equation  8. 19  can  be  expanded  and  rewritten 
as  a  polynomial  in  x ^  . 
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Proof  (Cont. ) 


X  =  (-  l)k(n  -  k)C.n  £  (-  i)m_L_  Ck  Xln‘m 
n,  k  '  k  ^  A  n  -  m  m  1 


(8.21) 


If  the  "A"  decision  is  relaxed  slightly  to  admit  k  +  1  indi¬ 
cations  of  conditions  N  ,  the  additional  probability  of  false 
alarm  is  a  probability  of  exactly  k  +  1  indications  of  con¬ 
dition  N  . 


X  ,  -  -  X  =  C,n  (1  -  xjk+1  x,n~k~l 
n,k+l  n,k  k+1  r  1 


(8.  22) 


This  can  also  be  written  in  terms  of  a  polynomial  in  . 


kn,k+l 


(-  i)k+1(„  -  k)  c“ 


(- 1) 


„  k  n-  m  -  n 
C  x.  +  C,  .x, 
m  1  k+1  1 


n-k-1 


(8.23) 

The  exact  form  in  Eq.  8. 23  has  been  chosen  to  make  it  as 
similar  as  possible  to  Eq.  8.  21.  In  order  to  establish  the 
theorem,  the  highest  power  of  x^  in  Eq.  8.21  must  be  elimi¬ 
nated  by  canceling  it  with  the  highest  power  in  Eq.  8.  23.  This 
highest  power  corresponds  to  the  summation  index  m  =  0  . 
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Proof  (Cont. ) 

This  dictates  that  we  try  adding  (k  +  l)/n  of  Eq.  8.  23  to 
Eq.  8.21. 


Since  the  manipulation  on  ^  is  identical  to  those  for 
X  .  ,  the  theorem  has  been  established. 

One  additional  fact  about  the  binomial  ROC  curves  that 
develop  from  repeated  Luce  observation  mny  be  noted.  Pass  a  pure 
power  ROC  through  the  Luce  vertex.  The  parameter  value  for  this 
pure  power  ROC  is 

Xj  =  yxA  A  -  (In  Xj)/(ln  yj)  (8.25) 


The  first  vertex  above  the  point  (0,0)  on  <sach  of  the  binomial  ROC 
curves  corresponds  to  saying  "A"  if  and  only  if  there  have  been  no 
indications  of  condition  N  .  The  coordinates  of  this  vertex  are 
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therefore  the  n-th  power  of  the  coordinates  for  the  Luce  vertex. 

Vn,  0  =  (xin  ’  O  <8'  26) 


Therefore,  this  vertex  also  falls  on  the  pure  power  ROC  curve  with 
index  A. 


(8.  27) 


Consider  an  observer  whose  complete  one- observation  ROC  curve  is  a 
pure  power  ROC.  If  this  observer  records  decisions  made  on  a  number 
of  independent  observations,  and  forgets  everything  else  about  the 
observation,  then  his  individual  decisions  act  as  if  they  came  from  a 
Luce  ROC.  If  he  then  uses  these  several  decisions  to  reach  a  terminal 
decision,  his  terminal  decision  will  be  no  better  than  one  he  could  obtain 
from  a  single  observation,  if_  his  final  cru  ricr,  demands  confirmation 
of  the  condition  SN  on  each  observation.  In  order  to  use  the  multiple 
decisions  to  reach  a  better  terminal  decision,  he  must  employ  not-too- 
lax  a  criterion  for  the  individual  decisions,  and  not- too- strict  a  criterion 
for  the  terminal  decision.  This  behavior  of  the  initial  point  on  the  poly¬ 
gon  ROC  curves  is  shown  in  Fig.  8.  2  for  the  same  Luce  and  Green 
double  threshold  ROC  used  in  Fig.  8. 1.  The  first  part  of  the  polygon 
ROC  falls  below  the  pure  power  ROC,  whiie  the  remainder  falls  above 
the  pure  power  ROC. 

The  work  above  has  been  very  detailed,  and  is  the  type 
of  analysis  one  wc*  Id  use  when  the  number  of  observations  leading  to  a 
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terminal  decision  is  very  smaii.  When  the  number  of  observations 
leading  to  a  terminal  decision  is  large,  a  description  of  a  continuous 
ROC  character  that  approximates  the  binomial  ROC  character  would  be 
convenient.  The  two  possible  log  likelihood  ratios  that  occur  in  the 
Luce  ROC  character,  and  the  corresponding  jumps  in  the  integrated 
ROC  character  are  sketched  in  Fig  8.  3. 


a' 


a). 


Fig.  8.  3.  Sketch  for  Luce  ROC 
character 


The  equations  for  these  quantities  in  terms  of  the  Luce  ROC  point 
(Xj.  yj)  are  given  in  Eq.  8.  28. 


\  =  In  ^(1  -  yjj/d  -  Xj)j  ,z2  =  ln(y1/x1) 

"1  =  {d-y1)0  -*,))' 6  .  <*ihyS 


(8.  28) 


The  binomial  ROC  character  for  n  observations  is 


dlln(z)  -  C^11  k  at  z  =  (n-  k)  z^  +  kz2  (8.  29) 


Equation  8. 29  is  not  the  familiar  binomial  probability  equation  unless 
the  sum  of  cuj  and  Wg  is  one.  Since  an  expression  that  is  the  binomial 
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probability  function  would  be  useful,  multiply  and  divide  by  the  sum 
of  o>i  plus  il>2  to  obtain  Eq.  8.  30 


U)1  n-k  <j)0 

dn  (z)  =  [u.'1+(u0]n  c£  ( - - - )  ( - — ) 

n  1  2  k  v  u>«  +  uJ  1 


W1  +  ^2 


W1  +  w2 


<8.  30) 


The  standard  approximation  for  the  binomial  probability  function  is  a 
normal  probability  density  with  mean  value  and  variance  given  by 
familiar  equations.  Since  we  want  to  use  this  approximation  for  z, 
and  not  for  the  integer  value  k,  we  must  multiply  by  the  Jacobian  of 
transformation.  This  is  indicated  here 


[^1+^2  ]"n  TTn(z)  ~ 


1 


(k-  m); 
2a2 


v/  2tt  ct 


ok 

dz 


N  (8-31> 


where  the  values  for  the  parameters  m  and  a2  are  given  in  8.  32. 


m  =  n 


2 

+  u.’ 


-) 

2 


2  “1^2 
ct  =  n, 


(8.32) 


This  is  the  continuous  ROC  character  which  will  be  used  to  approximate 
the  discrete  character  of  the  actual  binomial  ROC.  To  obtain  this 
equation  in  terms  of  the  variable  z,  use  Eq.  8.  29  for  z  in  terms  of  n 
and  k  to  do  the  transformation.  After  the  very  straightforward  algebra, 
the  resulting  form  for  the  approximating  ROC  character  is  given  in 
Eq.  8.33. 


7T  (z )  1  [^  +  (x'2 


(z  ;  n/3  )2 
j  2nf) 

r2nnD  6 


nzj  x  z  v  nz^ 


w,  ux 

D  =  (Z2  ‘  Zl)2  (Wj  +  i^2)2 


?  = 


Vi_1YL2_ 

(wl  +  W2)Z 


(8.33) 


The  parameter  D  has  the  role  of  a  variance  term,  while  the  parameter 
will  be  interpreted  as  a  skewness  parameter.  If  the  range  of  z  were 
infinite,  the  normal  ROC  character  would  have  to  have  a  mode  of  zero, 
which  would  preclude  the  use  of  Eq.  8.  33  except  for  p  values  of  zero. 
However,  the  approximating  ROC  character  of  Eq.  8.  33  applies  only  to 
the  bounded  range  of  z  values  that  can  be  obtained  from  the  original 
Luce  character  in  n  observations.  If  the  original  Luce  character  is 
nonsymmetric,  this  resulting  range  will  also  be  nonsymmetric. 

Therefore,  Eq.  8.  33  is  interpreted  as  a  metastatic  normal  ROC  character. 

If  the  original  Luce  ROC  character  is  symmetric,  the 
skewness  parameter  p  will  be  zero,  and  the  range  of  the  approximating 
normal  character  will  also  be  symmetric.  This  is  shown  in  Eq.  8. 34. 


Xj  =  1  -  y^  P  =  0  ,  D  =  (In  )  =  (*2)  (8.34) 


OEM 
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The  parameters  D  and  ft  apply  to  the  individual 
distribution;  the  factor  n  appears  separately  in  Eq.  8.  33.  These 
quantities  D  and  ft  are  the  parameters  of  the  original  Luce  ROC  curve 
that  are  relevant  in  studying  the  limiting  form  of  the  binomial  ROC  result¬ 
ing  from  many  similar  observations. 

Table  8. 1  presents  three  examples  of  Luce  ROC  curves 


yl 

Z1 

Z2 

“i 

con 

2 

ft 

ft 

V  D 

D 

.0100 

.0918 

-.0862 

2.215 

.  949 

.030 

-.164 

-.042 

.157 

,  .  1586 

.5000 

-.5200 

1.150 

.  649 

.273 

-.026 

-.034 

.  580 

.3085 

.  6915 

-.  806 

.  806  ' 

.  462 

.  462 

0 

0 

.  650 

Table  8. 1.  Limiting  binomial 
examples 


and  the  values  of  D  and  ft  evaluated  for  them.  The  three  points 

chosen  for  the  Luce  ROC  all  fali  on  a  normal  ROC  with  index  d  of 

unity.  The  first  was  picked  as  x^  =  .01,  a  very  strict  criterion. 

The  second  was  chosen  at  =  .  50.  The  third  was  chosen  at  the 

negative  diagonal.  A  column  for  the  ratio  of  ft  to  the  square  root  of 

D  has  been  included.  The  parameter  ft  is  the  skewness  parameter, 

giving  the  amount  that  the  z  mode  shifts  for  each  observation.  The 

ratio  of  ft  to  the  square  root  of  D  indicates  the  mode  shift  in 

standard  deviation  units.  The  highest  value  of  the  parameter  D  results 

from  the  symmetric  Luce  ROC.  The  fact  that  this  is  only  65  percent 
2 

(roughly  — )  of  the  original  d  is  familiar  to  those  who  have  worked 
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with  detectors  based  upon  clipped  correlation  devices  sometimes  known 
as  polarity  coincidence  comparators  (Ref.  36).  The  performance  is 
fairly  insensitive  to  the  departure  from  the  negative  diagonal  until  the 
departure  becomes  extreme.  For  very  strict  criteria,  the  parameter 
D  is  very  much  lower  than  the  original  normal  ROC  parameter  d, 
and  the  resultant  binormal  ROC  will  be  poorer  than  if  a  more  sym¬ 
metric  original  position  had  been  used. 

8.  3  Fisher-Tippett,  Doubly  Exponential  Character 

In  Section  3.3.5,  the  Fisher-Tippett  distribution  was  used  to 
obtain  an  ROC  character.  The  form  of  this  character  is  repeated  here. 

tt(z)  =  Aeaz'^e  a  >.5,  j3  >0  (8.35) 

The  probability  distribution  associated  with  this  character  arises  in 
studios  of  the  distribution  of  the  largest  of  a  number  of  independent 
random  variables.  In  that  application,  the  two  multipliers  of  z  are 
identical.  This  section  will  treat  a  slight  generalization  of  this  char¬ 
acter,  where  the  multipliers  of  z  need  not  be  identical.  This 
character  is  shown  in  Eq.  8.  36. 

tt(z)  =  a  eflZ  ’  <*>'  '>*  5  (8.36) 

The  coefficients  a  and  y  may  be  either  positive  or  negative,  but 
they  must  both  have  the  same  sign.  Otherwise,  the  ROC  character 
would  not  approach  zero  for  large  magnitude  z.  The  coefficient  /3  is 


always  positive. 
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Examination  of  the  first  two  derivatives  of  the  exponent  show 
that  this  ROC  character  is  unimodal.  Let  £  (z)  denote  the  exponent 
in  Eq.  8.  36. 

£  (z)  =  a z  -  fleyZ 
£  '(z)  =  a  -  (3y  eyZ 

£^(z)  =  -j3yn  eyz  n  >  2  (8.37) 

The  mode  of  the  ROC  character  is  at  the  z  value  for  which  the  first 
derivative  is  zero. 

i  a 

mode  z  :  ~  In  ^  (8.  38) 

The  second  derivative  of  the  exponent  is  everywhere  negative,  confirm¬ 
ing  that  the  Eq.  8.  38  yields  a  mode,  and  not  a  relative  minimum. 

£"(z)  =  -13 y2  eyZ  <  0  (8.39) 

From  ROC  Curve  to  Character,  The  easiest  demonstration  of  the  ROC 
curves  corresponding  to  the  characters  of  Eq.  8.  36  is  to  exhibit  the 
ROC  curves  and  then  to  show  that  they  have  this  character.  The  incom¬ 
plete  gamma  function  is  defined  in  Eq.  8.37. 

P(a,t)  =  /  Xa_1e"XdX  a  >  0  (8.40) 

The  ROC  curve  will  be  a  comparison  of  two  incomplete  gamma  functions 
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at  different  parameter  values  a,  but  with  the  same  cut  t.  The  ROC 
curves  a^e  given  parametrically  in  Eq.  8.  41.  The  notation  used  gives 
the  two  parameter  values  in  terms  of  their  mean  and  difference. 


X(t)  =  1  -  P(m  -  fi,  t) 


Y(t)  =  1  -  P(m  +  6,  t)  K 
0  <  5  <'  m 


(8.41) 


To  obtain  the  ROC  character  first  find  the  probability 
density  functions  for  the  dummy  variable  t,  and  then  transform  to  the 
log  likelihood  ratio.  Since  the  ROC  curve  is  given  in  terms  of  integrals, 
the  probability  density  functions  are  simply  the  integrands. 


fit  i 

x~\  p*  1  /  c  \ i m —  5  "  ^  “t 

N)  =  T  (m  -  6  )t  e 

(8.  42) 

f(t  1 

!SN)  =  r  1(m  +  6  )tm^*  _1  e_t 

(8.  43) 

These  are  Pearson  III  probability  density  functions  in  the  variable  t.  The 
likelihood  ratio  is  formed  by  dividing  Eq.  8.  43  by  8.  42. 

ez  =  t^  T  (m  -  6)/  F  (m  +  6)  (8.  44) 


The  likelihood  ratio  has  a  Pearson  III  probability  density  function,  but  z 
will  not.  To  obtain  the  probability  density  functions  for  z,  solve  Eq.  8.44 


for  t  • 


j 


i 


i 


233 


t  = 


1 


r  (m  +  6)1^ 
r  (m  -  6) J 


z 

2F 

e 


(8.  45) 


It  will  be  convenient  to  have  a  symbol  for  the  constant  factor  in  Eq.  8.  43. 
Call  this  constant  /3 


= 


p  >0 


(8.  46) 


Rewrite  Eq.  8.  45  as 


z 


and  differentiate  to  obtain  the  Jacobian  of  transformation. 


z 

dt_  _  26  _  t 

dz  26  e  -  26 


(8.  47) 


(8.  48) 


The  root  product  density  function  for  t  is 

|  f(t  IN)  f(t  |SN)  j  '  5  =  [  r  (m  -  6)  r(m  +  5)p5  tm‘le'1 

(8.  49) 


Since  the  Jacobian  of  transformation,  Eq.  8.  48,  contains  a  single  power 
of  t,  absorb  this  into  the  root  product  density  before  substitution  for 
t  according  to  Eq,  8.  44.  The  result  is 


viz)  =-jj  [r(m-  5)  r(m  +  5)]’5 


m 


mz 

23"  - 

e  e 


z 

23- 


< 

s. 


(8.  50) 
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To  simplify  this  equation  to  the  general  form  of  Eq.  8.  36,  merely 
identify  the  constant  factor,  A,  and  the  two  multipliers  of  z,  a  and  } . 

A  =  j  [F(m  -  6  )  r<»  +  5 )]  ‘  ®  /3  m 

a  =  r  »  a>.5,  y>0  (8.51) 

The  restriction  that  the  parameter  6  be  between  zero  and  m  converts 
to  the  restriction  that  the  coefficient  a  be  greater  than  one- half  and 
that  y  be  positive. 

Whenever  an  ROC  character  is  nonsymmetric,  a  second 
ROC  character  and  ROC  curve  are  obtained  by  reversing  the  z-axis. 
Specifically,  replace  z  by  -z  everywhere  in  the  first  ROC  character, 
and  replace  X  by  1  -  Y  and  Y  by  1  -  X  in  the  equation  for  the  first 
ROC  curve.  When  this  is  done  for  the  doubly  exponential  ROC  character 
of  Eq.  8.36,  another  doubly  exponential  ROC  character  is  obtained 

z  -*•  -z  X  -  1  Y  Y  -  1  -  X  (8. 52) 

-yZ 

n(z)  -  A  e~QZ  "  ^  a  .  5,  0  >  0,  y  >  0  (8.53) 

The  coefficients  are  the  same  as  those  given  in  Eq.  8. 51,  and  are 
positive.  The  two  minus  signs  can  be  absorbed  into  the  terms  a  and 
y,  giving  an  ROC  character  for  which  the  new  a  and  y  parameters 
are  both  negative,  but  the  coefficient  /3  is  still  positive.  The  ROC 
curve  is  given  parametrically  in  terms  of  the  incomplete  gamma  function. 
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From  ROC  Character  to  ROC  Curve.  Whenever  a  doubly  exponential 
ROC  character  is  encountered,  the  parameters  necessary  for  the 
incomplete  gamma  functions  used  in  the  ROC  curve  formulae  are  easily 
obtained.  Th<?  m  parameter  is  the  ratio  of  the  two  multipliers  of  z, 
a  and  y.  These  coefficients  will  either  be  both  positive  or  both  nega¬ 
tive,  and  hence  their  ratio  is  positive.  The  difference  parameter,  6, 
is  given  solely  in  terms  of  the  y  parameter.  Although  there  are  only 
two  degrees  of  freedom  in  the  Fisher- Tippett  Doubly  Exponential 
Character,  there  are  three  coefficients  in  the  exponential.  One  must 
therefore  make  sure  that  the  ,1  parameter  has  the  appropriate  value. 
This  is  summarized  in  Eq.  8.  55. 


Check:  $ 


[  T(m  +  6)  /  r(m  -  6)] 


(8.  55) 


Tables  of  the  incomplete  gamma  function  are  not  always 
available.  The  incomplete  gamma  function  is  related  to  the  chi-square 
distribution  whenever  the  parameter  values  m  +  $  are  integers  or  half 


integers.  The  relation  is  that  the  chi-square  value  is  2t  and  the 
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degrees  of  freedom  are  twice  the  incomplete  gamma  parameter. 


P(m  +  M)  =  P(x* 


2t 


v  =  2m  +  26) 


m,  6  are  both  odd  quarter  positive  integer 

or  are  half  positive  integer  or  positive  integer 


(8.  56) 


j 

Whenever  the  degrees  of  freedom  of  the  chi-square  distribution  are 
even  integers,  the  probability  may  be  written  in  terms  of  a  finite 
power  series,  a  polynomial,  times  a  simple  exponential. 


1  -  P(m  +  6  ,t) 


m-1  +  6  .n 
-t  v  -  t 


n=0 


n  ! 


*\ 


m,  6  are  both  half  positive  integer  or  both 
positive  integer 


(8.  57) 


J 

Tables  for  the  incomplete  gamma  function  and  the  chi-square  are  nor¬ 
mally  available  for  only  a  low  number  of  degrees  of  freedom.  Typically, 
the  largest  tables  available  will  terminate  at  thirty  deg'ees  cf  freedom. 


Approach  to  Normal.  As  the  number  of  degrees  of  freedom  become 
large,  the  Chi-Square  distribution  approaches  a  normal  distribution. 

The  following  approximation  is  usually  suggested 

1-P{y2|  *>)  a  $  (-v2x*  +  v2y-l),  v  large  (8.58) 

The  table  in  Cramer  (Ref.  £)  suggests  v  of  thirty  is  sufficiently  large 
to  use  this  approximation,  while  the  NBS  Handbook  (Ref.  6,  26.  4.13) 
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suggests  v  larger  than  100.  ROC  curves  deal  with  a  comparison  of 
two  distribution  functions.  Our  exper  ence  has  been  that  such  approxi¬ 
mations  may  be  used  at  much  lower  va.ues  of  parameter  in  a  comparison 
than  would  be  allowed  were  one  evaluating  a  single  distribution.  The 
approximation  of  Eq.  8.  58  used  in  evaluating  the  ROC  curves  of  Eq.  8.  51 
will  necessarily  yield  a  normal  ROC  curve.  The  parameter  d  will  be 
given  by  the  square  of  the  difference  of  the  two  terms  involving  the 
two  degrees  of  freedom.  When  both  of  the  degrees  of  freedom  are 
large,  and  their  difference  is  small  compared  to  their  average  value, 
s^me  standard  approximations  make  a  further  simplification  in  deter¬ 
mining  the  normal  ROC  parameter.  The  manipulations  are  omitted  but 
the  result  is  given  below. 

5  =  (vim  -1  +  26  -  v  4m  -  i  -  26)  s  46 2  m  *  =  (ay)  *  (8.57) 

Equation  8.  59  strongly  suggests  that  no  single  parameter  in  the  doubly 
exponential  character  can  be  interpreted  as  the  quality  parameter  for 
this  distribution.  Rather,  detectability  will  be  inversely  proportional 
to  the  product  of  the  two  multipliers  of  z,  a  and  y. 

Since  the  ROC  character  is  nonsymmetric,  the  exact 
ROC  curve  will  be  nonsymmetric.  When  the  ROC  curve  is  plotted  on 
normal- normal  paper,  the  nonsymmetry  should  be  manifested  as  a 
slope  different  from  one.  It  would  be  advantageous  to  have  a  value 
obtainable  from  the  character  which  would  be  simply  related  to  this 
slope.  In  Carver's  tables  (Ref.  28)  for  the  Pearson  m  distribution,  the 
skewness  of  the  incomplete  gamma  function  is  given  in  terms  of  the 
parameter  a. 
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a  g  =  skewness  of  P(a,  t)  =  2  a 


(8.  60) 


Although  the  ROC  equations  use  two  incomplete  gamma  distributions 
with  different  skewness,  under  the  condition  that  m  is  much  larger 
than  the  difference,  6,  a  is  of  the  order  of  2/ V  m  .  No  quantitative 

O 

measure  is  implied.  It  is  implied  that  the  skewness  should  disappear 
as  m  is  increased. 

The  approximation  given  in  Eq.  1.  59  groups  together 
those  ROC  curves  with  the  same  ratio  of  62  to  m.  If  the  role  of  6 


is  replaced  by  a  new  parameter,  d  ,  using  the  substitution  6  =  .  5\/md 

2i  3. 


the  ROC  character  is  rewritten  as 


jt(z)  -  A  e 


£ 


z  -  tie 


~7md 

a 


(8.61) 


Collecting  Eqs.  8.  41  and  8.  56,  the  ROC  curve  for  the  above  ROC 
character  in  terms  of  the  cni- square  distribution  is 


=  1  -  P(y2  =  2t  v  =  2m  +  ) 


(8.62) 


For  tables  which  give  the  upper  tail  area,  which  is  one  minus  the  chi- 
square  probability, 


X  =  Q(xa  =  2t|y  =2m+  Vmda) 


(8.  63) 


Holding  d  constant,  it  can  be  shown  that  behaves  like 
m  minus  one-eighth  as  m  approaches  infinity. 

A 


For  m  values  much  larger  than  the  detection  index  d  ,  j3  may  be 

cl 

considered  equal  to  m.  In  this  limiting  form,  the  double  exponential 
ROC  character  approaches 


m  -  oo  ,  jr(z) 


V  md_ 
me  a 


(6.  66) 


Examination  of  the  derivatives  of  the  exponent  for  this  limiting  form, 
shows  that  the  mode  has  shifted  to  zero  and  that  the  derivatives  of  order 
three  and  larger  rapidly  approach  zero.  The  second  derivative  of  the 
exponent  is  the  only  derivative  that  does  not  approach  zero. 
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The  power  series  approximation  to  the  exponent  is  dominated  by  the 
second  term. 


4  (z)  -  4  "(0) 


(8.  6 


This  is  the  exponent  for  the  normal  ROC  character. 

Using  Eq.  8.  63,  ROC  carves  were  evaluated  for 
vdn  -  1,  2,  and  3,  and  all  possible  small  m  values  yielding  integer 
degrees  of  freedom.  These  are  shown  in  Fig.  8.  4.  As  can  be  seen 
from  the  plots,  the  grouping  of  the  parameters  m  and  6  provided  by 
Eq.  8.  59  was  fairly  effective.  That  grouping  was  based  on  an  approxi¬ 
mation  to  the  chi-square  distribution  normally  considered  valid  for 
degrees  of  freedom  of  the  order  of  30  to  100  or  greater.  The  actual 
values  for  the  degrees  of  freedom  used  in  plotting  Fig.  8.  4  are  given 
in  Table  8.  2. 


d 

a 

nr. 

5 

^SN 

"N 

1 

1 

0.  50 

3 

1 

2.25 

0.75 

6 

3 

4 

1.00 

10 

d 

6.25 

1.25 

15 

9 

1.50 

21 

4 

4 

1 

12 

Il'SiSi 

9 

2 

24 

12 

9 

4 

3 

14 

2 

9 

4.5 

27 

_ 1_ 
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The  normal  ROC  curves  with  parameter  Vd  =  1,  2,  and  3  lie 
slightly  below  th?  double  exponential  character  ROC  curves,  crossing 
them  in  the  neighborhood  of  x  =  .  02.  The  development  in  this  subsection 
was  suggested  by  the  standard  approximation  to  the  chi-square  distri¬ 
bution  for  large  degrees  of  freedom.  This  approximation  indicated 
that  the  proper  way  to  consider  the  ROC  character 

v(z)  =  a  eaz  '  l3  eY  (8.36) 

is  to  consider  the  product  of  the  two  z  multipliers,  ay,  as  the  reciprocal 
of  the  approximating  normal  index,  and  to  consider  the  latio  of  these 
two  z  multipliers,  a/y,  as  a  skewness  index.  By  a  sequence  of 
approximations,  it  was  established  that  in  the  limit,  as  m  =  a/ y 
approached  infinity,  that  the  ROC  curves  do  become  symmetrical 
(indicated  by  the  z  =  0  point  falling  on  the  negative  diagonal),  and  the 
ROC  character  does  approach  normality  with  the  index  equal  to  the 
reciprocal  of  a', .  ROC  curves  based  on  available  tables  indicated  that 
this  same  viewpoint  of  the  parameters  in  the  ROC  character  is  valid 
for  small  integer  m,  as  well  as  when  m  approaches  infinity. 

Other  Cases.  The  cases  considered  numerically  showed  a  slope  greater 
than  one  on  normal- normal  paper,  contrary  to  most  of  the  examples  that 
we  have  considered.  The  reverse  double  exponential  ROC  character  of 
Eq.  8.  50  would  show  the  corresponding  slope  less  than  one,  since  these 
ROC  curves  would  be  the  mirror  image  of  those  shown  in  Fig.  8.  4. 
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In  the  above  subsection,  the  concern  was  with  cases  whete 
both  parameters  representing  degrees  of  freedom  became  large.  A 
particular  example  that  appeared  earlier,  in  Section  4.  4,  was  the 
ROC  curve 


Y  =  X(1  +  Jin  X  |)  (8.69) 

This  is  a  particular  case  of  the  double  exponential  character;  namely, 
when  a  is  y  +  .  5.  This  fixes  m  -  6  at  unity,  and  the  basic  equations 
for  the  ROC  curves  are 


X(t)  =  1-  P(l,t)  (8.70) 

Y(t)  =  1  -  P(2m  -  l,t)  (8.71) 

This  is  a  case  where  Eq.  8.  57  can  be  used  effectively,  if  2m  -  1  is  an 
integer. 

.  2m-2  n 

X(t)  -  e’\  Y(t)  =  e_t  ^  (8-  72) 


The  dummy  variable  t  can  be  removed  by  solving  the  simple  X(t) 
equation,  and  substituting  into  the  second  equation.  The  result  is 


Y 


• +  (im-iyy 


In  X 


2m-  2 


) 


(8.73) 


The  first  nine  such  ROC  curves  are  plotted  in  Fig.  8.  5. 


CHAPTER  IX 


ADDITION  OF  ROC  CHARACTERS 


This  chapter  deals  with  the  ROC  character  addition 
theorem,  and  with  two  families  of  two-parameter  ROC  characters.  The 
character  addition  theorem  says  that  if  a  number  of  regular  ROC  charac¬ 
ters  are  combined  in  a  weighted  sum,  then  the  resultant  ROC  curve  is 
formed  by  using  the  same  type  of  weighted  sum  over  the  original 
regular  ROC  curves. 


7i(z)  =  Yj  ck  7 Ik(z)  =>  X(z)  =  V  ck  XR(z) 

Y(z)  =  X  °k  Yk(z) 


(9.1) 


The  ROC  characters  treated  in  this  chapter  are  the  type 
containing  a  power  of  z,  and  an  exponential  term  in  z.  The  two  para¬ 
meters  in  each  family  are  the  power  of  z,  and  the  coefficient  in  the 
exponential.  The  first  family  considered  in  Section  9.  2  is  the  Pearson  in. 

Pearson  IE  Type:  v(z)  =  C  zp  e’Z/  k  (9.2) 

The  power  parameter,  p,  may  be  any  real  value  greater  than  -1.  The 
exponential  is  a  simple  exponential,  that  is,  it  is  linear  in  z.  The  second 
family  considered  in  Section  9.  3  is  a  form  of  the  Halsted  family  men¬ 
tioned  in  Chapter  I.  These  latter  ROC  characters  are  related  to  the 
Hermite  polynomials,  and  derivatives  of  the  normal  density  function. 
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H  Type: 


Czne-(z-z0)72k 


(9.3) 


This  type  of  ROC  character  is  just  being  developed  at  this  time,  and 
the  power  of  z  is  restricted  to  being  an  integer  greater  than,  or  equal 
to,  one.  The  K-type  ROC  character  differs  from  the  Pearson  III  charac¬ 
ter  in  that  the  exponential  term  is  quadratic  in  z.  Both  types  of  ROC 
character  are  bounded  below,  and  unbounded  above;  that  is,  the  z-range 
is  from  some  negative  value  up  to  plus  infinity. 

The  addition  theorem  allows  the  expansion  of  either  or 
both  of  these  types  to  ROC  characters  which  consists  of  polynomials  in 
z.  times  either  a  linear  or  quadratic  exponential  term.  When  the  equations 
for  the  ROC  curves  in  terms  of  the  variable  z  have  been  developed,  one 
may  immediately  obtain  the  ROC  curve  for  such  a  generalized  charac¬ 
ter. 


There  is  no  restriction  of  the  use  of  the  character  addi¬ 
tion  theorem  to  combining  characters  of  the  same  general  family.  Any 
types  of  character  may  be  added  together. 

9.  1  Character  Addition  Theorem 

This  section  states  and  proves  the  character  addition 
theorem.  The  statement  and  the  proof  are  in  terms  of  the  integrated 
ROC  character,  to  obtain  the  maximum  generally.  It  will  therefore 
apply  to  ROC  curves  that  are  differentiable,  and  have  ordinary  ROC 
characters;  it  will  also  apply  to  polygon  type  ROC  curves,  with  straight 
lines  joining  the  vertices,  as  well  as  mixtures.  The  key  mathematical 
point  in  the  proof  is  the  ability  to  interchange  a  finite  summation  with 
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integration. 

ROC  Character  Addition  Theorem  :  Let  n  (z)  be  the  integrated  ROC 

z),  Yk(z)),k  =  1,  2,  .  .  .  n. 

Let  be  any  positive  numbers  whose  sum  is  one.  Then 

n 

n(z)  =  Yj  nk^z^  (9- 4) 

k=  i  k  k 


characters  for  the  regular  ROC  curves  (X^( 


is  an  integrated  ROC  character,  the  ROC  curve  is  regular  and  is  given  by 


n 


XU)  =  l  ck  XJz)  Y(z)  =  V  ck  Y  (z) 
k  =  1  k  k  =  1 


(9.5) 


Proof : 

The  four  requirements  for  a 
an  integrated  ROC  chaiacter 
given  in  Chapter  in,  and  are 

(1)  the  lower  limit  is  zero 

(2)  monotone  growth 

(3)  unit  "N"  value 

(4)  unit  "SN"  value 


nonnegative  function  to  be 
of  a  regular  ROC  curve  were 


11  (-CO)  = 

:  0 

(3.6) 

dn  (z)  > 

0 

(3.7) 

f  -.  5z 

J  e 

dll  (z)  = 

1 

-00 

(3.8) 

00  c 

r  +.  5z 

J  e 

dll  (z)  = 

1 

-00 

(3.9) 

Each  of  the  given  individual  integrated  ROC  characters, 
rik,  have  these  four  properties.  Does  the  newly  formed 
sum  of  Eq.  9.4  also  have  these  properties?  If  the  individ¬ 
ual  integrated  ROC  characters  are  all  zero  at  z  -  -  x, 
then  the  sum  will  also  be  zero.  Similarly,  if  each  of  the 
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individual  integrated  ROC  characters  exhibits  monotone 
growth  (not  necessarily  strict  monotone  growth),  then  a 
finite  sum  does  also.  To  verify  Eqs.  3.  8  and  3.  9  for  the 
newly  formed  sum  one  need  merely  note  that  the  definite 
Stielijes  integral  is  a  linear  operator  to  both  functions. 
Therefore  the  integral  with  respect  to  the  sum  is  the 
sum  of  the  integrals.  Since  the  coefficients  c^  add  to 
one,  the  sum  of  Eq.  9.  4  will  satisfy  the  properties  3.  8 
and  3.  9. 

The  notation  for  the  ROC  curve  in  terms  of  the  single 
parameter  z  implies  the  dual  parameter  of  a  decision 
based  on  likelihood  ratio.  Specifically,  whenever  the 
integrated  ROC  character  has  a  jump,  the  associated 
straight  segment  of  the  ROC  curve  is  given  in  terms  of  a 
randomizing  parameter,  r.  The  complete  equation  for  the 
ROC  curve  in  terms  of  the  parameter  z  is  actually  given 
in  terms  of  a  cut  along  the  z  axis,  p,  and  the  randomizing 
parameter,  r,  whenever  it  is  needed. 


y(fj,  r) 
'xO.  r) 


00 


_  j  e±.  5z 


±.  5 B  /,,v  V  5,2  im\ 

+  re  H  u>  (£)  +  Ij  e  (z) 

W  v  A  ® 


±.  5z 


z  ^  p 


(9.6) 


Most  cf  this  present  work  has  dealt  with  the  first  integral 
alone,  since  for  most  ROC  characters  the  jump  function 


f  r**%**Si«wfe*«  ,-***  * 
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go  is  zero  everywhere.  Each  of  the  original  n  ROC 
curves  are  given  by  an  equation  such  as  Eq,  9.  6.  To 
determine  the  ROC  curve  for  the  integrated  ROC  character 
of  Eq.  9.4  substitute  the  equation  for  the  new  integrated 
character  into  Eq.  9.  6. 


Y(ft  r) 

r) 


f  e**  5z  Yj  ck  \  (z)  dz 

/I  k  k  k 


+  re 


5/3  y 


c,  , 
k  77k 


0)  +  >;  e 

z  >  /3 


±.  5z 


),  c,  oo  ,  (z) 
f-  k  77k  ' 
k 

(9.7) 


In  the  first  term,  the  interchange  of  the  order  of  summa¬ 
tion  and  integration  is  allowed  because  the  sum  is  finite. 
The  second  term  dealing  with  the  jump  at  the  point  /3  con¬ 
tains  only  one  summation  and.  therefore,  any  reordering 
of  the  terms,  is  valid.  The  third  summation  deals  with  one 
finite  sum  and  one  possibly  infinite  sum.  However,  since 
all  of  the  terms  involved  are  positive,  if  the  sums  converge 
in  any  order  they  will  converge  in  any  other  order.  There¬ 
fore.  the  summation  over  k  may  be  taken  outside  of  the 
expression  to  obtain  Eq.  9.  8. 
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+ 


V  ±.  5z 

L  e 

z>  (3 


(9.8) 


The  term  inside  each  bracket  is  the  pair  of  symmetric 
expressions  for  the  vertical  and  horizontal  coordinates 
of  the  ROC  curve,  and  The  proof  is  complete. 


9 .  2  Pear son  III  ROC  Characters 

In  Section  3.  3.  4  the  Pearson  III  character  was  obtained 
from  the  Pearson  III  probability  distribution.  The  form  of  the  ROC 
character  is 


7T(Z) 


1 

r(p+i) 


(z-zQ)p  e 


Bfl 

B-l 


B>  1,  p>  -1,  Zq  =  -(pfl)  In  B 


(9.9) 


In  this  section  explicit  expressions  for  the  ROC  curve  are  obtained  when 
the  parameter  p  is  either  an  integer  or  when  twice  p  is  an  integer. 

This  latter  is  usually  referred  to  as  half- integer  order.  In  each  case 
the  steps  will  be  to  display  an  ROC  curve  in  terms  of  a  dummy  parameter 
t.  obtain  the  relation  between  t  and  z,  and  check  to  be  sure  the  ROC 
character  is  truly  Pearson  in. 

Integer  Order.  Consider  any  nonnegative  integer  n,  and  the  following 
ROC  curve. 
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P  =  n  >  0 


•Bt  y  (Bt)k 

L  v,  i 

k  =  0 


,  n  tk 

r1  y  jl. 

Li  lrf 

k  =  0 


t  >  0 


(9. 10) 


E)th  X  and  Y  are  unity  when  t  is  zero,  and  decrease  toward  zero  as  t 
appro? '■hes  infinity.  Therefore,  X  and  Y,  as  usual,  are  treated  as 
one  minus  the  distribution  function  for  t.  The  probability  density  func¬ 
tions  for  t  are  the  negative  of  the  derivative  of  X  and  Y  with  respfect 


to  t.  Proceeding  formally,  the  derivative  of  X  with  respect  to  t  is 


jy  n.  n  i^k  ,k- 1  — .  n  .  c  , k 

dX  -Bt  v  B  t  „  -Bt  v'  B  t 

~dF  “  e  L  7k- iy  “  Be  La  -p — 
ai  k  =  1  k  =  0  K- 


(9.11) 


Collect  the  two  sums,  first  changing  indices  so  that  the  coefficients  of 
a  single  power  may  be  determined,  to  obtain 


n(  n-1  r+1  r  n  nk+l  k 

-Bt  \'  B  t  v  B  t 

e  -J  r>  ~  Li  k' 

r  =  0  r-  k  =  0 


Bn+1tn  -Bt 
~  e 


(9.12) 


Therefore,  the  density  function  for  t  under  condition  N  is 


f(tiN)  =  -y-1  lne"Bt 


(9. 13) 


Since  the  equation  for  Y  is  similar  to  that  for  X  with  B  set  equal  to 
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f(tfSN)  -  ~  tn  9_t  (9. 14) 

z 

The  ratio  of  these  two  equations  gives  the  likelihood  ratio,  e  . 

ez  =  B-(n+1>e(B-1)t  (9.15) 

Solving  for  z, 

z  =  (B-l)  t  -  (n+1)  In  B  (9. 16) 

By  Eqs.  9.  13  and  9. 14,  the  variable  t  has  a  Pearson  III  density  function. 
Since  z  is  a  linear  transformation  of  t,  it  too  will  have  Pearson  III 
density  functions  under  both  condition  N  and  SN.  Explicitly,  t  is 

1  =  (“B rr)'  zo  =  -(n+1)  In  B  (9.17) 

Any  of  the  several  possible  routes  to  obtain  the  ROC  character  will 
reobtain  Eq.  9.  9. 

It  is  therefore  established  that  Eq.  9.  10  is  the  ROC  curve 
for  a  Pearson  III  ROC  character  of  integer  order.  Eq.  9. 16  gives  the 
explicit  relation  between  z  and  the  variable  t. 

Half- Integer  Order.  Consider  any  value  for  p  which  is  one- half  larger 
than  an  integer.  The  claim  is  that  Eq.  9. 18  is  the  corresponding  ROC 
curve. 
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p  =  n  +  .  5  >  - .  5  X 

Y 

If  the  value  for  p  is  -.5,  so  that  n  would  take  on  the  value  -1,  the 
summation  is  omitted.  The  function  4>  (  )  is  the  normal  distribution 

function,  and  the  range  of  t  is  from  zero  to  infinity.  Differentiate  X 
with  respect  to  t  in  two  steps.  First  consider  the  term  involving  the 
normal  distribution  function.  The  detailing  of  differentiation  is  given 
in  Eq.  9. 19 


=  2-24)  (vTfflt  ) 


^5Ti+  >:  4S 


,k+.  5 


k  =  0 


r  k+1.  5 


(9.18) 


■  2-2$(v2t)  +  k^0  rkrs) 


n  ,k+.  5 

V  t  -t 

/i  “  ’  i  r  \  G 


(9.19) 


Tup  square  root  is  squared  in  the  normal  density  function,  leaving  a 
linear  exponential  term  multiplied  by  t  to  the  power  minus  one-half. 
The  differentiation  of  the  sum  is  subject  to  the  same  collapsing,  or 
telescoping  that  occurred  for  integer  order,  with  the  exception  that  two 
terms  are  left  affer  the  cancellation  instead  of  one. 


f 

8 


r 


I-,*  n  ,r„,k  +  .  5 

a  -Bt  v  (Bt) 

dt  "e  k“0  r(k+i.  5) 
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-Bt  -Bt  f  _ 

^TTiT-^  ft  e  ^  r(k+.  5) 


k  =  0 


TIE+1.  5) 


Qk+.  5,k-.  5 

D  t 


=  e 


-Bt 


n 

■l 


gk+.  5tk-.  5 


n  _k+l.  5.k+.  5 
V  B  t 

k^0  r(k+l.  5)  k^0  r(k+.  5) 


=  e 


-Bt 


qI1+1.  5  tn+.  5  g.  5  t~.  5 
r(n+l.  5)  r(.  5) 


(9.  20) 


The  two  remaining  terms  contain  the  desired  large  power  of  t,  and  a 
term  looking  very  much  like  that  in  the  last  line  in  Eq.  9.  19.  Indeed, 
if  the  gamma  function  is  evaluated  at  one-half,  it  is  simply  the  square 

root  of  77. 

r(.  5)  =  /I  (9.  21) 

In  the  sum  of  Eqs.  9. 19  and  9.  20,  these  similar  terms  will  cancel. 

(9.  22) 


,  Qn4T.  5  ,  n+.  5  R 

t(tlN>  -  ^(1-X)  =  T5W  e 


The  density  function  for  t  under  the  condition  SN  is  obtained  by  setting 
B  equal  to  one. 


n+.  F 

rit'SN)  - 


(9.  23) 


Divide  Eq.  9.  23  by  9.  22  to  obtain  the  likelihood  ratio. 

(9.  24) 

The  logarithm  of  the  likelihood  ratio,  z.  is  linear  wiih  the  variable,  t 


ez  _  B-(n+l.  5)  e(B-l)t 


J 
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and,  therefore,  will  have  the  same  type  of  density  function  as  t.  Since 
these  are  both  Pearson  III,  there  is  no  need  to  detail  the  familiar  route 
and  reobtain  the  ROC  character,  Eq.  9.  9,  for  half-integer  order. 

This  ha>  t  ‘'tablished  the  equations  for  the  ROC  curves 
for  Pearson  IE  character  for  integer  and  half  integer  order.  Other  forms 
of  these  equations  may  b  obtained  in  terms  of  the  chi-square  distribution, 
and  the  incomplete  gamma  function.  The  ROC  curve  is  given  in  terms  of 
a  comparison  of  iwo  points  along  either  of  these  distributions,  using  the 
same  number  of  degrees  of  freedom.  This  is  contrasted  to  the  case  of 
the  Fisher -Tippett  Doubly  Exponential  Character,  which  was  obtained 
from  the  incomplete  gamma  function,  or  chi-square  distribution,  by 
reading  at  the  same  argument  but  for  two  different  degrees  of  freedom. 

The  most  skewed  of  ihe  characters  considered  in  this 
section  is  that  with  p  =  -.5.  The  ROC  curves  for  several  B  values  are 
shown  on  normal-normal  paper  in  Fig.  9. 1.  There  is  a  special  coordi¬ 
nate  system  that  makes  these  curves  plot  as  straight  lines.  It  is  based 
on  the  error  function,  erf  t,  and  is  used  to  display  the  same  ROC  curves 
as  before  in  Fig.  9.  2. 

9.  3  H-Type  ROC  Character 

The  character  under  study  in  this  section  consists  of  a 
power  of  z,  times  a  quadratic  term  in  z  in  the  exponential.  The  general 


form  is 


.  ^.4  i*  •  v 
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-(/.  -  /  )2/  8a8 

ii 0/ ;  C  </.  -  /.Q)  e  ,  z  >  zQ.  n>  0 


(9.  25) 


The  term  is  the  minimum  value  for  z  and  will  be  negative.  If  the 
quadratic  in  the  exponential  were  expanded,it  would  contain  both  a  term 
in  z2  and  a  term  in  z.  The  constant  value,  C,  is  a  function  of  both 
the  power  n  and  the  detection  index  a  „  No  general  expression  for  this 
constant,  C,  or  the  value  of  z^  in  terms  of  n  and  a  is  known  at  this 
writing.  The  exponential  with  the  quadratic  term  can  be  expressed  as  a 
normal  density  function,  and  the  form  of  the  constant  C  changed  to  make 
subsequent  manipulation  simpler.  The  following  form  of  the  ROC  charac¬ 
ter  will  be  used 


7r(z)  = 


(9.  26) 


To  obtain  the  equations  for  the  ROC  curve,  multiply  ihe  character  by 

i  5? 

e  ’  .  and  integrate.  The  integration  will  be  simplest  with  the  change 

variable 


z0  dt  1 

Za  ’  dz  2a  ’  *  — 


(9.  27) 


The  multiplier  necessary  for  the  integration  can  be  expressed  in  terms 
of  a.  t.  and  z^. 


±  5z 
e 


±at  ±  .  5zq 


e 


(9.  28) 
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Reversing  the  usual  route,  the  density  functions  for  t  are  obtained  from 
the  corresponding  density  functions  for  z,  by  multiplying  Eq.  9.  28  by 
9.  26,  and  by  the  Jacobian  of  transformation. 


±at  ± 


A  e 


5V 


5  of 


i  n  . 

t  e 


5t‘ 


/  m 


(9.  29) 


Of  the  four  terms  in  the  exponential,  the  three  not  involving  z^  form 
a  perfect  square.  When  this  is  done,  the  densities  can  again  be  expressed 
in  terms  of  a  normal  density  function. 


./.  |SN\  .  ±-5z0fn 

lytj  =  Ae  t  0(t  +  a)  (9.30) 

The  equations  for  the  RCC  curve  can  then  be  formally  expressed  by 
integrating  the  density  functions. 


Y 


/  tn0(t-a)dt 
T 


-.  5zn  oo 

X  =  Ae  j  tn  0(t+a)  dt 
T 


(9.31) 


At  this  point  the  symmetric  form  for  these  equations  has  to  be  abandoned 
in  order  to  obtain  simpler  integrals.  Make  a  change  of  variables  in  each 


of  the  integrals  of  Eq.  9.  31  to  obtain  a  single  letter  argument  for  the 

,f  t 

normal  density  function. 
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Y  -  Ae  U  f  {t+af  0(t)  dt 
T-£V 


5z  oo 

X  =  Ae  u  f  (  t-af  0(t)  dt 
T+o 


(9.  32) 


The  derivatives  of  the  normal  density  function  can  be 
expressed  in  terms  of  a  polynomial  times  the  normal  density  function. 
These  polynomials  are  known  as  Hermite  polynomials.  Using  tables 
of  Hermite  polynomials  one  may  develop  formulae  for  the  integral  of 
a  single  power  times  the  normal  density  function.  Such  a  table  is  given 
here. 


/ 

0(t)  dt  = 

*(t) 

/ 

tp(t)  dt  = 

-0(t) 

/ 

t20(t)  dt 

=  0  (t)  -  t0(t) 

/ 

t30(t)  dt 

eg 

+ 

N 

1 

II 

f 

t4  0(t)  dt 

=  3®(t)  -  (t3  + 

3t)0(t) 

f 

t5  0(t)  dt 

=  -(t4  +  4t  2  + 

8)  0{t) 

J 

t6  0(t)  dt 

=  15*(t)  -  (ts 

+  5ts  +  1 5 1)  0(t) 

Table  9. 1  Special  Integrals 

The  equations  of  9.  32  may  be  integrated  by  expanding  the  term  (t±«)n  as 
a  polynomial  in  t.  The  coefficients  will  depend  upon  a,  and  whether 
the  sign  is  plus  or  minus.  By  using  the  table  of  special  integrals,  the 
equations  for  each  ROC  curve  may  be  reduced  to  a  term  involving  the 
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normal  distribution  function,  and  a  polynomial  times  the  normal  density 
function.  The  values  of  the  polynomial  times  the  normal  density  function 
at  the  upper  limit,  plus  infinity,  is  zero.  The  value  for  the  normal 
distribution  at  the  upper  limit  is  unity.  Therefore,  the  upper  limit  will 
contribute  at  most  a  constant  term.  The  rest  of  the  equation  for  the 
ROC  curve  will  depend  upon  the  evaluation  of  the  indefinite  integral  at 
the  lower  limits,  t  ±  a  The  resulting  form  for  the  ROC  curve  will  be 


Y  =  C. ,  [  1  -  $  (t  -  a)  1  +  l  (t  -  a)  0{t  -  a) 

X  =  CQ  1 1  -  $  (t  +  a)  j  +  Qn_  1  (t  +  a)  b(t  +  a)  (9.  33) 

a>  0  ,  t  >  0 


The  ROC  curve  and  ROC  character  have  been  graphed 
for  the  simplest  case,  when  n  is  one.  The  ROC  character  is  shown  in 
Fig.  9.  3.  The  lower  scale  is  the  dummy  parameter  t.  Five  z  scales 
are  shown.  As  the  parameter  a  increases,  the  ROC  character  will  be 
effectively  spread  over  much  more  of  the  z  axis.  The  mode  of  the  ROC 
character  is  negative,  and  effectively  moves  farther  to  the  left  as  detecta¬ 
bility  increases.  The  equations  for  the  ROC  curve  in  terms  of  the  normal 
distribution  and  density  for  n  =  1  are  given  below 


n  =  1 


Y  =  ^o(  1  -  4>  (t  -  «)]  +  0(t  -  a)^l  [  (a)  +  0(a)] 

X  =  ^-o{  1  -  $  (t  +  a)  |  +  0(t  +  o?)^  j  (-0!$  (-a)  +  0(ar) ) 


(9.34) 


The  ROC  curves  for  p=  1  and  2  are  displayed  on  normal- normal 
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paper  in  Fig.  9.  4.  They  appear  to  be  nearly  straight  lines,  with  slope 
less  than  one.  For  a=  1,  the  slope  is  s  =  .  85  and  for  a-  2,  the  slope 
is  .74. 

9.  4  Discussion  on  Approximations 

The  addition  theorem  may  be  used  to  advantage  to  obtain  an 
ROC  curve,  for  an  ROC  character  by  fitting  the  given  character  with  a 
weighted  sum  of  characters  with  known  ROC  curves.  If  the  nature  of 
the  tail  portion  of  the  given  character  is  known  precisely,  it  may  be 
matched  with  ROC  characters  with  similar  tail  oehavior.  The  Pearson 
III  class,  which  includes  the  simple  exponential  of  the  power  ROC,  all 
exhibit  a  linear  exponential  tail.  The  conic  ROC  class  also  has  a  linear 
exponential  tail,  with  coefficient  fixed  at  -1.5.  The  H-type  ROC 
character,  as  well  as  the  normal  and  metastatic  normal,  have  a  qua¬ 
dratic  exponential  tail  behavior. 

The  field  of  approximations  has  been  barely  studied  in  the 
present  work.  It  has  been  the  purpose  of  this  present  work  to  provide 
the  relation  between  the  ROC  curve  and  an  analytic  form,  the  ROC 
character,  and  to  present  a  variety  of  families  of  ROC  curves.  The 
author  intends  to  expand  the  ideas  presented  in  this  chapter  to  further 
enrich  the  collection  of  ROC  characters  with  known  HOC  curve  equations. 

The  area  of  obtaining  ROC  curves  and  characters  from  experi- 
mental  data  is  worthy  of  research.  Especially  useful  would  be  the 
development  of  techniques  for  successive  approximations,  whereby 


limited  data  could  be  used  to  obtain  a  first  ROC  curve  and  character 
and  subsequent  data  could  be  used  to  refine  the  first  approximation. 


CHAPTER  X 


SELECTED  TOPICS 

10.  1  Character- Free  Measures  of  ROC  Quality 

When  the  description  and  evaluation  of  an  ROC  curve  is 
just  op/1  part  of  an  experiment,  or  one  part  of  a  mathematical  analy¬ 
sis,  there  is  a  strong  practical  demand  to  reduce  the  description  of 
detection  performance  from  the  single  function  ROC  to  a  single  num¬ 
ber.  When  the  purpose  of  the  detection  is  well  defined,  this  can  be 
done  by  stating  the  degree  to  which  the  detection  performance  achieved 
its  purpos  .  For  example,  in  binary  communications  operating  with 
a  symmetric  channel,  the  likelihood  ratio  cut  is  uniquely  established 
at  one,  and  the  probability  of  error  is  the  single  numerical  quantity 
of  interest. 

One  of  the  major  purposes  of  this  present  work  was  to 
reduce  the  description  of  the  ROC  curve  to  a  two-step  description. 

The  first  step  is  the  determination  of  the  family  to  which  the  ROC 
curve  belongs.  The  second  step  is  the  evaluation  of  the  one  or  two 
parameters  necessary  to  describe  the  particular  curve.  The  specific 
parameters  to  be  evaluated  depends  on  the  nature  of  the  character  for 
the  particular  ROC  family. 

In  this  section  various  measures  of  ROC  quality  are 
considered  that  have  been  proposed  for  use  independent  of  the  particu¬ 
lar  character  of  the  ROC  curve.  We  call  such  measures 
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"character-free.  "  The  definition  of  the  measure  is  independent  of  the 
character  of  the  ROC.  However,  the  values  obtained  for  such  mea¬ 
sures  will  depend  on  the  particular  character  of  the  ROC. 

10.  1.  1  D.  M.  Green's  Theorem.  A  particular  type  of 
psychophysical  experiment  is  known  as  the  multiple  presentation, 
multiple  alternative,  forced  choice  experiment,  or  simply,  "forced 
choice"  experiment.  The  description  which  will  be  given  here  is  the 
forced  choice  in  time  experiment.  There  are  other  forms  of  experi¬ 
ment  with  equivalent  analysis.  In  the  two  alternative  forced  choice 
experiment,  two  intervals  in  time  are  marked  for  the  observers  atten¬ 
tion.  A  signal  will  appear  in  one  but  not  both  of  these  time  intervals. 
The  observer  is  required  to  decide  which  interval  contains  the  signal. 
In  the  general  M-alternative  forced  choice  situation,  M  intervals 
are  marked  off,  and  the  signal  will  appear  in  one  and  only  one  of 
these  intervals.  A  completely  symmetric  forced  choice  experiment 
presents  the  signals  in  each  interval  with  equal  probability,  and  the 
subject  is  asked  to  obtain  the  maximum  number  of  correct  identifica¬ 
tions.  The  single  number  score  for  his  performance  is  the  fraction  of 
identifications  that  are  correct.  The  corresponding  analytical  mea¬ 
sure  for  a  theoretical  situation  is  the  maximum  probability  of  correct 
identification. 

Green's  Theorem  (Ref.  29)  relates  the  score  for  a  two- 
alternative  symmetric  forced  choice  experiment  to  the  ROC  curve. 
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Experimentally,  the  ROC  curve  is  obtained  in  a  single  presentation 
two- alternative  situation.  Green's  Theorem  is  that  the  two- alternative 
forced  choice  score  will  be  the  area  under  the  ROC  curve  plotted  on 
linear  paper. 

-  Area  under  the  ROC  curve  (10.  1) 

A  general  formulation  for  the  probability  of  correct 
identification  for  the  M  alternative  symmetric  forced  choice  situa¬ 
tion  in  terms  of  the  ROC  curve  can  be  readily  obtained.  Consider  an 
observer  with  a  given  ROC  curve  which  measures  his  ability  to  distin¬ 
guish  between  signal  in  noise,  SN,  from  a  background  of  noise  alone, 

N.  If  this  ROC  curve  is  regular,  then  it  corresponds  to  any  one  of  a 
number  of  decision  axes.  Consider  any  such  decision  axis,  and  label 
it  t.  When  the  cause  is  N,  denote  the  decision  axis  value  by  t^;  when 

the  cause  is  SN,  denote  the  random  variable  by  t.,.T.  Consider  the 

bN 

observation  upon  which  the  subject  bases  his  identification.  It  consists 
of  M  -  1  values  of  the  decision  axis  due  to  N,  and  only  one  value  re¬ 
sulting  from  SN.  To  maximize  the  number  of  correct  identifications, 
the  subject  should  choose  the  largest  value  of  t  as  corresponding  to 
the  interval  with  the  signal  in  it.  The  probability  that  he  will  be  cor¬ 
rect  is  the  probability  that  the  value  of  t  due  to  SN,  tg^,  is  greater 
than  all  of  the  other  observed  decision  axis  values. 


PM(C)  =  Prob  (all  M  - 1  *N  <  tgN) 


(10.2) 
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Assume  for  convenience  that  t  has  a  probability  density  function 
under  the  condition  SN;  then  P^(C)  can  be  written  in  terms  of  an 
integral  as  follows:  The  probability  of  correct  identification  condi¬ 
tional  to  any  specific  value  of  tg^  is  the  product  of  the  probabilities 
that  each  of  the  t^  is  below  the  tg^  value.  This  individual  proba¬ 
bility  is  the  distribution  function  for  t  under  condition  N.  For  the 
symmetric  presentation  case,  all  t^  variables  have  the  same  distri¬ 
bution.  P^(C)  is  obtained  by  averaging  over  all  possible  tg^  values. 

OC  ^  -j 

PM(C)  =  /  F  "  (tlN)  f(tiSN)  dt  (10.3) 

-cc 

It  is  convenient  to  convert  this  integral  to  the  more  general  Stieltjes 
integral. 

pM(C)  =  /  FM~ 1  (ti  N)  dF(tlSN)  (10.4) 

M  0 

No  matter  which  of  the  decision  axes  we  hypothesized  for  the  observer, 
the  distribution  function  under  N  is  simply  1  -  X;  the  distribution 
function  under  SN  is  1  -  Y. 

pM(C)  =  f1  (1-X)M_1  d(l-  Y)  (10.5) 

M  0 

A  change  of  variables  from  1  -  Y  to  Y  involves  a  minus  sign  which 
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is  absorbed  in  reversing  the  limits  of  integration. 


Vc) 


=  /  (1-  X)M"  1  dY 


(10.6) 


Setting  M  equal  to  two  in  Eq.  10.  6  yields  the  two  ex¬ 


pressions 


1  1 
P9(C)  =  /  (1-  X)  dY  =  1  -  /  Xd  Y 
0  0 


(10.  7) 


Interpreting  the  integral  as  the  area  under  the  curve  of  the  integrand, 


P2(C)  =  Area  under  ROC  curve 

=  1  -  Area  above  ROC  curve 


(10.  8) 


Another  integral  which  is  equivalent,  since  it  also  computes  the  area 
under  the  ROC  curve,  is 


1 

P9(C)  =  f  YdX  (10.9) 

&  0 

Green’s  Theorem  gives  a  strong  geometric  relation, 
easily  understood,  which  relates  two  psychophysical  experiments. 

It  also  emphasizes  the  insensitive  nature  of  the  forced  choice  score 
on  the  detail  structure  of  the  ROC  curve. 

The  two- alternative  measures  applied  to  the  Luce  ROC 
curve  with  vertex  (xj,  y  ^),  to  the  normal  ROC  curve  with  index  d, 


and  co  the  exponential  ROC  curve  with  index  A,  are  given  in  Eqs. 

10.  10.  10.  11,  and  10.  13,  An  excellent  approximation  to  the  M-alter- 
native  probability  for  the  normal  ROC  curve  is  given  in  Eq.  10.  12 
(Ref.  17). 


Luce  (xj,  yj): 

P2<C> 

=  .  5  +  .  5(y1-x1) 

(10.  10) 

Normal,  d: 

P2<C> 

=  $(v  5d) 

(10.  11) 

Normal,  d: 

PM<C> 

^aM^d_  bM)’  =  M 

(10. 12) 

Exponential  (Power), 

A:  P2<C>  -  AA+ 1 

(10,  13) 

Green’s  Theorem  related  the  probability  of  correct 
identification  in  the  two- alternative  test  to  the  ROC  curve.  Every 
quantity  related  to  the  ROC  curve  is  also  related  to  the  ROC  character. 
The  following  theorem  will  establish  the  corresponding  relationship 
between  P2(C)  and  character- 

Any  real  Lg  function  has  an  autocorrelation  function. 
The  form  chosen  for  the  (unnormalized)  autocorrelation  function  of  the 
ROC  character  is 

oc 

0  (r)  =  f  ?r(z  +  .  5r)  7r(z  -  .  5t)  dz  (10.14) 

-ac 

It  will  be  shown  that  the  two- alternative  probability  of  correct  identi¬ 
fication  is  the  area  under  the  autocorrelation  on  the  positive  side  of 
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zero,  weighted  with  an  exponential  factor  which  emphasizes  large 
values. 

Theorem : 

p2(c)  =  f  e*57?Wr)dr  (10.15) 

Proof: 

Equation  10.  9  is  the  starting  point.  In  order  to  apply 
the  integration,  the  explicit  relation  between  corres¬ 
ponding  values  of  X  and  Y  is  needed. 

x  _ 

yip)  =  f  e+'  ^  tt(z)  dz,  -dx(/3)  =  e~  *  ir(i3)  dp  (10.16) 

When  these  are  formally  utilized  in  Eq.  10.9,  the  minus 
sign  on  the  differential  for  x  disappears  in  the  Jacobian 
of  transformation. 


P2<C>  = 

/  /  e'  5z  ir(z)  dz  e  '  n(/3)  d/3 

(10.  17) 

-x  /3 

P2(C)  = 

/  /  e’  5(z_/3)  v( z)  m  dzd/3 

(10.  18) 

-x  <  /3  <  z  <  x 

The  region  of  integration  is  over  the  half- plane  of  those 
values  of  z  which  are  greater  than  p.  Any  alternative 
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description  of  the  same  half-plane  can  be  used  as  well. 
A  change  of  variables  which  singles  out  z-]3  has  been 
selected. 


z  =  u  +  -  5t  u  =  .  5(z  +  ft) 

ft  -  u  -  .  5t  r  =  z  -  ft 


1 


(10.  19) 


The  half- plane  of  integration  is  now  described  as  the 
half- plane  with  positive  r,  and  u  ranging  from  -oc  to 

+oc. 


OC  oc 

P9(C)  =  /  /  e tt(u  +  .  5r)  rr(u  -  .  5r)  du  dr  (10.20) 

Z  0  -oc 


The  integral  over  u  is  the  autocorrelation  function  of 
the  character. 


P9(C)  =  /  e' 5r  0  (r)  dr  (10.15) 

2  0  ™ 

The  proof  of  the  theorem  is  complete. 

10.  1.  2  W.  W.  Peterson’s  Theorem.  The  basic  relation 
between  the  moments  of  the  likelihood  ratio  under  the  two  distributions, 
N  and  SW,  was  first  pointed  out  to  this  writer  by  W.  W.  Peterson. 
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Theorem? 

E(£nlN)  -  E(£ n" 1 ISN)  n  >  1  (10.21) 

This  is  a  direct  result  of  the  fundamental  theorem  that  the  likelihood 
ratio  of  the  likelihood  ratio  is  the  likelihood  ratio.  The  proof  of  Eq. 

10.  21  can  be  accomplished  simply  in  one  line  by  writing  down  the 
integral  definition  for  the  n-th  moment.  Factoring  out  one  £  term 
and  associating  it  with  the  differential  yields  the  desired  (n-  l)st  mo¬ 
ment  under  condition  SN. 

f  (n  dF(£  I  N)  -  /  £n_1  £dF(£lN)  =  f  £n" 1  dF(£  ISN) 

This  theorem  leads  to  a  basic  understanding  of  some 
calculations  on  likelihood  ratio  used  to  obtain  measures  of  ROC  quality. 
The  average  value  of  likelihood  ratio  under  noise  is  unity  because  the 
average  value  of  the  constant  one  under  condition  SN  is  unity.  The 
average  value  of  likelihood  ratio  under  condition  SN  is  the  second 
moment  under  condition  N. 

E(£  I N)  =  1  E(£2lN)  =  E(£ISN)  (10.22) 

This  means  that  the  variance  of  the  likelihood  ratio  under  condition  N 
will  be  equal  to  the  difference  of  the  means  under  the  two  conditions. 


oz(i  IN)  =  E(£  ISN;  -  1  =  E(£  ISN)  -  E(«  IN) 


(10.  23) 
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The  signal- to- noise  ratio,  snr,  for  a  linear  filter  is 
defined  as  tne  square  of  the  peak  output  shift  caused  by  the  presence 
of  signal,  compared  to  the  average  noise  power.  In  similar  fashion, 
many  authors  including  the  present  one,  will  define  the  snr  for  any 
kind  of  device  as  the  square  of  the  shift  in  the  expected  value,  divided 
by  the  variance.  When  applied  to  linear  filters,  for  situations  in  which 
a  signal  is  added  to  stationary  noise,  the  definitions  agree.  In  general 
detection  theory,  any  receiver  is  considered  a  likelihood  ratio  receiv¬ 
er  whose  output  is  a  fixed  order- preserving  transformation  of  likeli¬ 
hood  ratio.  Consider  for  the  moment  a  receiver  whose  output  is 
precisely  the  likelihood  ratio  of  the  input.  The  signal- to- noise  ratio 
for  this  output  is 


(snr)f 


[E(jMSN)  -  E(jg  IN)] 2 
o2(£1N) 


(10.  24) 


By  virtue  of  Eq.  10.  23,  this  signal- to- noise  ratio  is  also  equal  to  those 
quantities  in  Eq.  10.  23.  If  this  signal-to- noise  ratio  could  be  inter¬ 
preted  in  usual  engineering  fashion,  and  if  this  likelihood  ratio  is  the 
one  that  leads  to  a  terminal  decision,  then  an  order  of  magnitude  of 
the  quantities  involved  can  be  inferred.  In  most  engineering  applica¬ 
tions.  the  signal- to- noise  ratio  desired  is  of  the  order  of  ten,  or  a 
hundred,  or  a  thousand,  or  greater.  If  the  variance  of  the  likelihood 
ratio  under  N  is  any  of  these  numbers,  while  the  expected  value  is 


fixed  at  unity,  then  the  distribution  of  £  must  be  skewed  to  the  right 
with  tremendous  upper  tail-  A  description  based  on  only  the  first  two 
moments  is  hardly  an  adequate  description  for  such  a  statistic.  There¬ 
fore  internal  consistency  dictates  that  Eq.  10.  24  be  considered  a  valid 
description  of  ROC  quality  only  when  its  value  is  smaller  than  one. 

In  an  attempt  to  compensate  for  the  highly  skewed  na¬ 
ture  of  the  distribution  of  likelihood  ratio  for  cases  with  large  detection 
quality,  W.  W.  Peterson  presented  the  following  argument.  For  a  nor¬ 
mal  ROC  curve,  the  measures  of  Eqs.  10.  23  and  10.  24  are  all  related 
to  the  index  d  in  exponential  manner. 

Normal  ROC  E(f  ISN)  =  ed  (10.25) 

The  index  d  is  therefore 

d  =  Cn  [1  +  a2  (J2  IN)]  (10.  26) 

When  a  signal  is  added  to  white  Gaussian  noise,  and  applied  to  the 
matched  linear  filter,  the  distribution  of  likelihood  ratio  is  log  normal, 
and  d  agrees  with  the  accepted  value  of  signal- to- noise  ratio.  He 
therefore  suggested  that  if  one  wishes  to  base  a  quality  statement  on 
the  first  two  moments  of  likelihood  ratio,  tiiat  one  should  do  so  by 
computing  the  variance  of  £  under  N  and  applying  Eq.  10.  26. 

10. 1. 3  Comparison  of  Quality  Measures.  In  this  sec¬ 
tion  four  measures  of  detectability  are  applied  to  four  types  of  ROC: 
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normal,  rectangular  character,  two-line  symmetric  Luce  character, 
and  the  exponential  character. 

Kulback,  Leibler,  and  Jeffries  (Ref.  31  )  have  used  a 
measure  which  they  call  "the  divergence  between  hypotheses,  "  or  "the 
difference  in  mean  information  for  decision  per  observation  available 
from  the  hypothesis  space.  "  It  is  the  difference  between  the  means 
of  the  logarithm  of  the  likelihood  ratio. 

J(1:0)  =  E(zlSN)  -  E(zlN)  (10.27) 

The  second  measure  of  performance  will  be  found  used  in  many  papers 
in  the  physical  theory  of  detectability,  called  output  signal- to- noise 
ratio  of  a  decision  device.  It  is  the  difference  of  the  means  of  the 
logarithm  of  the  likelihood  ratio,  squared,  divided  by  the  variance  of 
the  log  likelihood  ratio. 

(snr)  =  J2(l:0)/a2(z!N)  (10.28) 

z 

This  form  is  used  when  dealing  with  a  decision  axis  which  is  known  to 
be  a  linear  translation  of  the  z  variable,  but  in  which  the  constant 
has  been  lost  during  the  analysis  manipulations.  It  is  justified  because 
the  difference  of  the  means  removes  any  translational  constant,  and  the 
division  by  the  variance  will  remove  any  multiplicative  factor.  A  third 
measure  of  detectability  was  suggested  by  W.  W.  Peterson. 


d^p  =  In  (1  +  or2(£  IN))  =  In  E(f  3 IN) 


(10.  29) 
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The  fourth  measure  of  detectability  is  the  variance  of  the  logarithm 
of  the  likelihood  ratio  taken  under  either  hypothesis. 

ct2(z  I N)  or  a2(z  iSN)  (10.30) 


The  four  ROC  characters  to  be  considered  are: 


d  z 

Normal,  d  >  0:  trXT(z)  «  e  -  e 

N  ® 


rR 


Rectangular,  R  >  1:  ffR(z)  "  2(1  +  R) 


I zl<  In  R 


Symmetric  Luce,  a  >  0:  u)^(z  =  ±a.)  =  (e  +  e  ) 

1 


- 1 


1  A+l  z 
1  ,  i ,  1  "  A- 1  2 


Exponential,  A  >  0:  7Tj,(z)  =•  ^-r  (^-)  e  z  >  -In  A 


Tabie  10.  1  lists  expected  values  and  exact  equations  for 
the  four  measures  of  detectability.  All  four  measures  of  detectability 
are  the  same  for  the  normal  ROC  curve.  The  four  measures  of  de¬ 
tectability  are  not  the  same  for  the  other  types  of  ROC  curves.  Let  us 
examine  the  approximate  value  of  these  measures  at  the  two  extremes 
--when  the  detectability  is  very  low,  and  when  the  detectability  is  high. 


^4 
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Character 

Index 

Limiting  Performance 

Rectangular 

R  -  1 

l  (In  R)2 

Sym  Luce 

a  -  c 

4a2 

Exponential 

A  -  1 

(A-l)2 

. 

Table  10.  2.  Low  detectability  limiting  formulae 

All  four  measures  differ  in  the  way  the  limits  are 
approached.  However,  for  small  signals  all  four  measures  agree.  If 
one's  interest  in  detectability  is  limited  to  a  problem  in  which  repeated, 
independent  and  similar  observations  will  be  taken,  then  any  one  of 
these  four  measures  of  individual  detectability  will  be  appropriate  for 
determining  the  ultimate  detectability.  In  predetermined  observation 
procedures  or  in  sequential  observation  procedures,  the  logarithm  of 
the  likelihood  ratio  is  the  sum  of  the  individual  z  values.  The  resul¬ 
tant  ROC  curve  for  fixed  size  decision  procedures  will  be  approximate¬ 
ly  normal,  with  index  d  equal  to  the  sum  of  the  individual  detection 
"d/'  values.  For  example,  one  may  validly  conclude  that  if  a  final 
decision  is  based  on  n  individual  decisions,  each  decision  yielding 
an  observable  output  with  log  likelihood  ratio  either  +2a  or  -2a,  then 
the  final  ROC  will  be  nearly  normal  and  the  detectability  index  will  be 
d  =  4n  a2. 

We  have  demonstrated  that  each  of  these  four  measures 
applies  to  the  normal  ROC  curve  independent  of  the  level  of  detectability, 


and  to  other  types  of  ROC  curve  (at  least,  for  a  small  class  of  ROC  j 
curves)  at  low  levels  of  detectability. 

If  one  could  similarly  conclude  that  for  asymptotically 
large  levels  of  detectability  all  four  measures  were  nearly  the  same, 
one  would  have  great  confidence  in  using  any  of  them.  However,  for 
each  character  these  measures  will  approach  different  limits  and  j 

j 

approach  them  at  different  rates. 


Table  10.  3.  High  detectability  limiting  formulae 

In  summary,  several  measures  of  detectability  have 
been  examined,  and  have  been  shown  to  lead  to  common  numerical 
evaluations  when  the  detectability  is  low.  The  normal  ROC  curve  is 
unique  in  that  all  four  measures  of  detectability  yield  the  same  value, 
d,  at  all  levels  of  detectability.  When  the  ROC  curves  near  perfection, 
the  four  measures  of  detectability  become  drastically  different.  Not 
only  do  the  measures  disagree  for  the  same  type  of  character,  but  the 
behavior  of  each  measure  is  different  for  different  characters. 
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Anyone  using  a  single  number  characterization  for  the 
ROC  c  urve  level  without  mention  of  the  form  of  the  ROC  character 
must  be  very  careful  to  know  the  use  that  will  be  made  of  his  one  num¬ 
ber  in  order  to  have  an  appropriate  measure  of  detectability. 

10.  2  A  Special  Case  of  Type-II  ROC 

In  certain  psychophysical  experiments  two  quality 
measures  have  been  obtained  simultaneously,  one  of  which  is  an  ROC 
curve  (Ref.  14).  The  first  such  experiment  known  to  this  writer  is  an 
experiment  of  Egan's  for  articulation  score  testing  (Ref,  32).  A  word 
was  randomly  selected  from  a  known  list  of  fifty  words,  mixed  with  noise, 
and  presented  to  a  subject.  The  subject  responded  to  each  utterance 
by  selecting  one  of  the  words  from  the  known  set  as  the  most  likely 
transmitted.  The  novel  part  of  the  experiment  was  that  the  subject 
was  asked  to  score  himself;  that  is,  to  indicate  which  responses  he 
fell  were  right  and  which  responses  he  felt  were  likely  to  be  wrong. 

An  ROC  curve  was  then  obtained  by  plotting  on  the  horizontal  axis  the 
proportion  of  words  accepted  as  correct,  which  were,  however,  wrong, 
and  on  the  vertical  axis,  the  proportion  of  words  accepted  as  correct, 
and  indeed  were  correct.  This  was  a  measure  of  the  listener's  ability 
to  detect  his  own  correct  responses  in  a  background  of  incorrect 
responses.  This  ROC  curve  is  a  "type- two”  ROC  curve,  the  distinc¬ 
tion  being  made  that  the  subject  was  indicating  something  about  his 
first  response.  The  first  response  was  measured  with  a  percent 
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correct  or  articulation  score. 

In  this  section  a  somewhat  simpler  experiment  leading 
to  two  responses  from  the  device,  or  subject,  * s  analyzed.  Both  re¬ 
sponses  will  be  evaluated  on  ROC  curves.  Consider  any  two-alterna¬ 
tive  single- presentation  experiment  as  the  primary  experiment.  As 
always,  it  is  not  important  in  ROC  analysis  that  one  causal  hypothesis 
be  background  noise  and  the  other  a  signal  introduced  into  the  noise; 
any  two  causes  may  be  used  as  long  as  there  is  a  correct  response 
"A”  corresponding  to  the  cause  SN.  and  alternative  response  ”B”  cor¬ 
responding  to  the  second  cause.  N.  The  response  to  the  first  detection 
experiment  will  be  subscripted  throughout  by  ”sub  one”;  that  is,  the 
ROC  curve  will  be  as  a  function  of  X^,  with  log  likelihood  z,, 
and  ROC  character  n^(z).  The  second  response,  leading  to  a  type- 
two  ROC  curve,  will  be  a  judgment  on  the  part  of  the  observer  as  to 
whether  the  first  response  was  correct  or  incorrect.  That  is,  the 
observer  is  asked  to  place  his  responses  into  two  categories  ,  the  high 
category  indicating  those  quite  likely  to  be  right,  and  the  low  category 
being  those  which  are  less  likely  to  be  right. 

What  is  the  observation  on  which  the  subject  bases  this 
second  response?  It  is  exactly  the  same  observation  on  which  he  based 
his  first  response.  The  observation  itself  is  not  different;  the  type  of 
response  is  different.  The  relevant  probability  densities  will  also  be 
different.  The  only  feature  of  the  observation  that  one  need  consider 
is  the  log  likelihood  ratio  Zy  Although  this  is  the  logarithm  of  the 
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likelihood  ratio  for  the  first  decision,  it  may  help  to  de- emphasize 
this  nature  of  the  observation;  therefore  replace  by  the  simple 
letter  "t"  as  the  observation  for  the  second  decision. 

The  type-two  ROC  curve  conditions  are  right,  R.  or 
wrong.  W,  corresponding  to  whether  the  first  decision  was  right  or 
wrong.  Two  ways  the  subject  would  be  right  would  be  detection, 

A  -  SN,  and  correct  rejection.  B-  N.  Two  type- two  ROC  curves  are 
to  be  developed,  the  first  conditional  to  the  response  A,  and  the  sec¬ 
ond  conditional  to  the  resnnnse  b.  This  is  done  for  three  reasons; 
one,  the  subject  knows  the  response;  two,  the  experimenter  or  grader 
knows  the  response;  and  three,  these  conditions  may  be  significantly 
different.  In  order  to  develop  type-two  ROC  curves  a  specific  point 
must  be  picked  as  operating  point  on  the  type- one  ROC  curve.  Let  /3 
designate  a  cut  in  the  axis,  and  the  corresponding  ROC  point  be 

<xr  V 

The  lype-two  ROC  curve  conditional  to  the  response  ”A:’ 
is  relevant  whenever  the  observation  t  is  greater  than  the  cut  value 
(1.  The  density  function  for  t,  conditional  to  being  right,  is  the  proba¬ 
bility  density  of  t,  given  the  joint  condition  A  •  SN.  Care  is  taken 
here  to  indicate  the  conditions,  so  that  the  proper  normalization  will 
be  included. 
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t  >  J  f(t  |R)  =  f(t | A,  SN) 

=  fit,  A|SN)/P(A|SN)  (10.31) 

-  e+'5t7r1(t)/Y1 

In  similar  fashion,  the  probability  density  for  t,  given  that  the  re¬ 
sponse  was  wrong,  is  the  probability  density  for  t,  conditional  to  the 
joint  condition  A  •  N. 

t  >  j3  f(t  1 W)  =  f  (t  j  A ,  N) 

-  f(t,A|N)/P(AlN)  (10.32) 

=  e‘*5tir1(t)/X1 

The  likelihood  ratio  relevant  for  the  second  type  of  re¬ 
sponse  is 


£(t)  -  f(t|R)/f(t!W) 

Using  (10.31)  and  (10.32),  Eq.  10.33  becomes 


t  ‘  0  z2  =  t  +  MXj/Yj) 


The  ROC  character  is 


7T2(z  I  A)  =  n 
X1 

(3  4-  111  rp- 

M 


(10.33) 


(10.34) 


(10.35) 


The  type- two  ROC  curve  conditional  to  the  response  ”A"  is  the  meta¬ 
static  ROC  curve  derived  from  the  type- one  ROC  considered  from 
the  point  (0.0)  to  the  point  Y^). 

The  type- two  ROC  curve  conditional  to  the  response 
"B"  is  similarly  derived  by  considering  t  values  less  than  j3.  The 
probability  density  for  t  conditional  to  a  wrong  response  is  the  proba¬ 
bility  density  for  t.  condition  to  the  joint  occurrence  of  the  response 
B  and  the  condition  SN. 

t  <  P  f(tlW)  =  f(tlB.SN) 

=  f(t,  BISN)/P(BISN)  (10.36) 

.  =  e+"  5t  «•  j(t)/(l-  Yj) 

Similarly,  the  probability  of  t  given  the  right  response  is 

t  <  /3  f(t!R)  =  f(t I B.  N) 

=  f(t,  B|N)/P(B|N)  (10.37) 

=  e'-^TTjCtJ/d-Xj) 

When  the  observable  t  is  less  than  the  relevant  likelihood  ratio  is 

£(t)  =  f(t|R)/f(tlw)  (10.33) 

The  value  of  z ^  and  character  are 
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1_Y1 

t  <  p  z2  =  “t  +  In  (10.38) 

1  —  ^ 

ir2(zlB)  =  ir^-z-lnjTY^^il-XjKl- Yx)  (10.39) 

-  /3  ln(  1  -  Y^)/ (‘1  -  Xj)  <  z  <  -  min  z  ^ 

+  ln(  1  -  Y1)/(l-  Xj) 

This  type- two  ROC  curve  is  the  metastatic  ROC  curve  obtained  from 
the  type-one  ROC  by  considering  the  arc  from  (X.,,  Yj)  to  the  extreme 
(1,  1)  point. 

These  two  type- two  ROC  characters  are  both  obtained 
from  the  type-one  ROC  character.  Graphically,  the  type-one  charac¬ 
ter  ir  cut  apart  at  the  point  corresponding  to  the  operating  point  /3, 
and  the  left  portion  reflected  about  the  cut  point,  that  is,  the  -oc  tail 
swung  around  toward  +cc.  The  appropriate  horizontal  translations 
and  vertical  magnifications  are  applied  to  yield  a  legitimate  pair  of 
ROC  characters. 

It  appears  that  three  conditions  are  both  necessary  and 
sufficient  to  make  the  two  resulting  type- two  ROC  characters  equal. 
The  magnifications  will  have  to  be  equal,  so  the  original  operating 
point  must  fall  on  the  negative  diagonal.  The  domains  of  z  must  be 
the  same  (and  utilizing  the  first  point  that  the  operating  point  is  on  the 
negative  diagonal),  the  cut  point  /3  must  be  zero.  Third,  and  most 


important,  the  original  ROC  character  must  be  symmetric  in  order 
that  the  mirror  image  7^(2  iB)  be  equal  to  ^(z!  A). 

For  many  reasons  an  analyst  may  wish  to  lump  two 
conditional  type-two  ROC  curves  together  as  a  single  curve.  This 
may  be  done  formally  by  simply  averaging  the  type- two  ROC  charac¬ 
ters.  These  two  characters  may  be  averaged  if  and  only  if  the  cor¬ 
responding  cuts  in  the  type- two  responses  are  not  conditional  to  the 
type- one  response.  That  is,  in  reaching  a  type- two  decision  the  ob¬ 
server  must  cut  the  z^-ax\s  at  a  single  point,  and  not  one  cut  condi¬ 
tional  to  the  A  response  and  a  second  cut  conditional  to  a  B  response. 
The  ROC  character  is  zero  outside  of  the  ranges  displayed  in  Eqs. 

10.  35  and  10.  39.  Using  p  and  q  to  indicate  p  =  P(SN)  and  q  =  P(N), 
the  a  priori  probabilities  for  the  type- one  conditions,  the  lumped  data 
type- two  ROC  curve  will  have  character 


77 2(z)  =  P(A)  7T2(zIA)  +  P(B)  7T2(z  I  B> 


PY1+qX 


1  -  PYj  -  qXj 


(10.  40) 


1-X, 

'il‘z'ln  Fy, 


For  a  symmetric  case,  where  the  type- one  character  is  symmetric 
and  the  observer  has  selected  the  symmetric  cut,  (3  =  0,  falling  on  the 
negative  diagonal,  the  type- two  ROC  curve  is 
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The  experimental  procedure  determines  if  a  specific 
decision  device  operates  according  to  this  optimum  theory  would  re¬ 
quire  the  responding  device  (subject)  to  operate  at  a  fixed  operating 
point  for  the  type- one  decision.  Either  a  number  of  fixed  operating 
points,  or  a  rating  scale,  may  be  used  to  obtain  the  type-two  ROC  curves. 

10 .  3  The  Multiple  Observer 

The  title,  "Multiple  Observer.”  comes  from  psycho¬ 
physics.  It  refers  to  the  situation  where  the  final  decision  maker  is 
aided  by  several  subordinate  decision  makers.  In  the  context  used  in 
this  present  work,  the  multiple  observer  has  been  generalized  to  in¬ 
clude  situations  when  the  subordinate  processors  may  or  may  not 
reach  decisions  themselves.  There  is  no  essential  restriction  to 
situations  involving  heman  observers.  An  electronic  system  with 
inputs  from  several  types  of  sensors  is  a  multiple  observer. 

Formally,  the  multiple  observer  is  considered  as  a 
terminal  decision  maker  whose  input  observation  consists  of  a  vector 
V  containing  the  outputs  of  the  n  subordinate  observers.  The  em¬ 
phasis  in  multiple  observer  studies  is  usually  on  those  situations  for 
which  the  outputs  of  the  subordinate  observers  are  of  different  nature. 
However,  all  outputs  are  influenced  simultaneously  by  the  presence 
or  absence  of  signal.  The  terminal  decision  maker  may  axso  be  one 
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of  his  own  subordinate  observers. 

Such  situations  are  prevalent  in  our  everyday  life.  A 
person  who  must  reach  a  decision  as  to  whether  it  will  rain  this  after¬ 
noon  or  not,  may  listen  to  a  weather  forecast,  talk  to  his  friends,  and 
observe  the  sky  himself.  However,  he  must  reach  the  final  decision 
"A",  to  act  as  if  there  will  be  no  rain,  or  ”B”  to  act  as  if  there  will 
be  bad  weather.  He  takes  his  own  observation  into  account  as  well  as 
those  of  his  friends  and  weather  specialists.  He  must  process  his 
own  visual  information  ir  the  appropriate  manner,  and  combine  that 
processed  output  appropriately  with  the  information  from  his  other 
sources. 

We  shall  make  two  assumptions:  first,  that  all  subor¬ 
dinate  observers  are  operating  under  the  same  condition,  N  or  SN; 
second,  that  inputs  to  the  final  decision  maker  are  statistically  inde¬ 
pendent.  Likelihood  ratio  theory  yields  what  the  proper  processing 
by  the  final  decision  maker  should  be,  and  what  his  ROC  character  is. 
He  should  compute  the  logarithm  of  the  likelihood  ratio  for  each  com¬ 
ponent  of  his  input,  and  add  these  to  obtain  the  logarithm  of  the  likeli¬ 
hood  ratio  of  his  total  observation.  His  ROC  character  is  the 
convolution  of  the  characters  of  the  subordinate  observers. 


V  =  (tr  t2, 


z(V) 


S  ’ 

k=l  K  K 


•  42) 
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The  likelihood  ratio  of  a  decision  can  be  considered  as  easily  as  the 
likelihood  ratio  of  any  other  kind  of  observable.  If  the  k-th  subordin¬ 
ate  observer  makes  the  decision  "A”,  and  his  ROC  operating  point 
(X,  .  Y,  )  is  known,  then  the  likelihood  ratio  of  the  observation  is  the 

K  K 

ratio  of  Y,  to  X.  .  This  is  the  ratio  of  the  likelihood  of  the  decision 
k  k 

”A"  under  the  condition  SN  compared  to  the  likelihood  of  the  decis¬ 
ion  "A”  under  the  condition  N.  Therefore,  each  subordinate  observer 
who  is  a  decision  maker  will  behave  as  if  his  ROC  is  a  Luce  ROC. 


^  c  ("A”,  "B"(,  zk("A")  =  ln(Yk/Xk) 

zk("B")  =  ln((l-Yk)/(i-Xk)) 


(10.  43) 


No  matter  how  different  the  types  of  observations  the 
subordinate  observers  make,  no  matter  how  different  their  ROC 
curves,  and  style  of  reporting  are,  the  formal  description  of  the  mul¬ 
tiple  observer  treats  each  of  the  subordinate  observers  similarly. 
That  is,  the  terminal  decision  maker  calculates  the  log  likelihood  ra¬ 
tio  of  the  subordinate  to  know  how  to  process  his  information,  and 
considers  the  ROC  character  of  the  subordinate  in  order  to  determine 
his  own  ROC  character. 

While  the  problem  of  the  final  decision  maker  is  fairly 
straightforward,  the  problem  of  the  subordinate  observer  who  must 
report  a  decision  is  much  more  difficult.  The  overall  optimum  be¬ 
havior  for  the  subordinate  observer  is  to  report  his  decision  value, 
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his  z  value,  to  his  superior.  (This  is  obviously  optimum  since  the 
superior  could  do  no  worse  than  process  this  decision  value  in  the 
same  way  that  the  subordinate  observer  does.)  The  problem  of  se¬ 
lecting  the  optimum  operating  point,  the  cut  value  on  the  z  axis,  for 
the  subordinate  observer  forced  to  make  a  decision  is  unsolved  ex¬ 
cept  for  a  few  special  cases.  Several  workers  have  suggested  that 
the  cut  be  chosen  to  maximize  the  Shannon  information  measure. 
Certainly  further  research  is  necessary  to  arrive  at  a  completely 
satisfactory  technique.  It  is  hoped  that  the  description  of  the  final 
decision  maker’s  performance  in  terms  of  his  ROC  character  may 
provide  a  new  approach;  the  subordinate  observer  may  set  his  goal 
as  the  choice  of  the  Luce  ROC  character  which,  convoluted  with  the 
other  ROC  characters,  will  yield  the  best  final  character. 

An  observer  who  reports  a  continuous  decision  axis  is 
called  a  continuous  observer,  and  one  who  reports  a  discrete  number 
of  decision  values  is  called  a  discrete  observer.  The  phrase  "Luce 
observer"  is  reserved  for  a  subordinate  observer  who  reports  a 
binary  decision.  The  resulting  ROC  character  for  situations  involv¬ 
ing  continuous  observers,  or  discrete  observers  who  report  many 
distinct  values,  is  best  handled  by  direct  application  of  convolution. 

In  this  section  two  examples  will  be  presented.  The  first  is  a  com¬ 
bination  of  one  Normal  and  one  Luce  observer;  this  is  treated  numer¬ 
ically.  The  second  example  is  of  one  exponential  observer  and  one 
Luce  observer;  it  is  possible  to  treat  this  situation  analytically. 
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Example:  Normal  Plus  Luce.  The  normal  ROC  character,  and  the 
ROC  curve  parameterized  by  the  variable  z  are  given  in  Eq.  10.  44. 


V-2* 


XN(z> 


YNfc) 


1 

V27rd 


<£(-.  5  Vd  -  z/  Vd) 


$(+•  5  Vd  -  z/ Vd) 


(10.  44) 


The  two  possible  z  values  and  the  corresponding  discrete  character 
values  for  a  Luce  observer  are 

ZA  =  ln(y3/xi)  •  wtt(zA)  "  V  X1  yl 

(10.  45) 

zB  =  ln(l-y1)/(l-x1)  ,  u>t(zb)  =  V  (1-XjKl-  yj 

The  convolution  of  these  ROC  characters  is  simple  because  only  two 
possible  z  values  occur  with  the  Luce  observer.  The  convolution  is 
therefore  the  sum  of  two  products. 

zr(z)  -  cujr(zA)  irN(z  -  zA)  +  co^Zg)  trN(z  -  zB)  (10.46) 

One  can  obtain  the  ROC  curve  by  one  of  two  routes.  The  formal  meth- 

i  5z 

od  is  to  multiply  the  ROC  character  by  e  *  and  to  integrate  from 
some  point  to  infinity.  The  other  is  to  verbalize  the  equation  as  fol¬ 
lows:  whenever  the  Luce  observer  reports  "A”  and  the  terminal 
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observer's  cut  value  is  z .  the  decision  maker  will  say  yes  whenever 
the  normal  observer  gives  a  value  greater  than  z-  z^.  Con¬ 
versely  whenever  the  Luce  observer  reports  "B",  the  normal  observ¬ 
er  must  give  a  much  stronger  indication,  namely,  z-  z_,  to  generate 

D 

a  terminal  "A"  decision.  Either  route  yields  precisely  the  same 
equations. 

Y(z)  =  y1  $(+.  5  Vd  +  ZpJ  Vd  -  z/  Vd)  +  (1  -  y^)  $(+.  5  Vd  +  Zg/Vcl  -  z/  Vd) 
X(z)  =  Xj  <&(-.  5Vd  +  z pj  Vd  -  z/  Vd)  +  (1-  x^)  4>(--  5Vcl  +  z^/Vd  -  z/  Vd) 

(10.  47) 

When  the  Luce  observer  is  operating  on  the  negative 
diagonal,  the  HOC  curve  and  the  ROC  character  are  symmetric.  This 
allows  °  slight  simplification  of  the  ROC  equations  and  some  further 
analytic  work. 

Symmetric  Luce:  x^  =  1  -  y^,  z0  -  -z^  (10.48) 

Whenever  the  normal  observer  is  sufficiently  superior  to  the  Luce 
observer,  the  ROC  character  of  Eq.  10.  46  will  have  a  single  maxi¬ 
mum  at  zero.  Under  the  converse  situation,  when  the  Luce  observer 
is  sufficiently  superior  to  the  normal  observer,  the  ROC  character 
will  be  bimodal.  The  precise  definition  of  "superior"  depends  on 
whether  z^  is  smaller  or  larger  than  the  square  root  of  d.  When 
the  Luce  observer  is  superior,  the  twin  modes  of  the  ROC  character 
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are  given  by  the  solution  of  Eq.  10.  49. 

z  zA 

>  Vd  ,  z  =  z ^  tanh  — g —  ( 10.  49) 

This  is  demonstrated  in  Fig.  10.  1,  where  the  ROC  characters  for  the 
final  decision  maker  are  plotted  on  semi- log  paper.  The  continuous 
normal  observer  had  an  index  of  d=  1.  Four  Luce  observers  were 
considered.  The  characteristics  of  these  four  are  listed  in  Table  10.  4. 


ZA 

0 

1.00 

1.  45 

2.95 

X1 

0.  50 

0.31 

0. 19 

0.  05 

*1 

0.  50 

0  69 

0.81 

0.95 

Table  10.  4.  Luce  z.  for  selected  points 


When  the  Luce  observer  operates  at  the  (.  19. .  81) 
point,  the  ROC  character  is  bimodal,  but  on  semi-log  paper  appears 
to  be  fairly  flat  topped.  The  ROC  curves  for  this  situation  are  plot¬ 
ted  in  Fig.  10.  2.  The  Luce  ROC  is  dotted,  since  the  only  point  used 
is  the  single  operating  point.  The  ROC  curve  for  this  observer  be¬ 
fore  he  was  forced  to  take  a  fixed  position  may  have  been  anything 
going  through  the  point  (.  19.  .  81).  In  the  neighborhood  of  the  nega¬ 
tive  diagonal  the  combined  Luce  and  norma1  observer  is  only  slightly 
be.. ci  than  the  Luce  operating  point.  However,  the  combination  is 
definitely  better  than  either  away  from  the  Luce  operating  point.  For 
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comparison  the  normal  character  for  the  curve  that  would  pass 
through  the  Luce  operating  point  is  drawn  together  with  the  multiple 
observer  ROC  character  in  Fig.  10.3(a).  That  part  of  the  character 
associated  with  die  ROC  region  appearing  on  one  percent  to  ninety- 
nine  percent  normal- normal  paper  is  also  indicated  in  Fig.  10.  3(a). 

When  the  Luce  observer  is  considerably  better  than 
his  co-worker,  the  normal  observer,  the  ROC  character  is  strongly 
bimodal.  This  is  best  seen  in  Fig.  10.  3(b),  plotted  on  linear  paper. 
This  strongly  suggests  its  origin  as  the  sum  of  two  displaced  normal 
density  functions.  When  the  ROC  curve  is  calculated,  that  region  of 
the  character  affecting  the  part  displayed  on  usual  normal- normal 
paper  is  basically  the  part  between  the  two  modes.  That  is  the  con¬ 
cave  section  of  the  character,  and  much  more  like  the  extreme  hyper¬ 
bolas  considered  in  Section  5.  2  than  like  the  normal  character.  The 
corresponding  ROC  curve  is  plotted  in  Fig.  10.  4.  This  ROC  displays 
much  more  of  a  "corner"  than  has  been  displayed  before  except  for 
the  Luce  ROC.  This  occurs  because  of  the  relative  minimum  on  the 
ROC  character,  which  places  very  little  probability  on  those  z  values 
near  zero.  As  a  result  the  slope  drops  from  five  (z  =  1.  6)  to  one 
(z  =  0)  with  only  a  0.01  change  in  false  alarm  probability. 

If  this  Luce  observer  represents  the  operating  point  on 
a  normal  ROC  curve,  the  index  would  be  Vd  =  3.  29  If  a  continuous 
normal  character  with  this  index  is  convolved  with  the  d  -  1  continu¬ 
ous  normal  character,  the  result  would  be  a  normal  ROC  with  index 


Fig.  10.  3.  Two  multiple  observer  ROC  characters 


cl  3.44.  The  d  1  observer  is  completely  overshadowed  by  the 
oeiver  observer.  However,  when  the  better  observer  reports  only  a 
decision  and  not  a  continuous  measure,  the  normal  observer  with 
d  1  becomes  important  in  improving  the  ROC  curve  away  from  the 
better  observer's  operating  point. 

Example:  Exponential  Plus  Luce.  The  following  work  is  restricted 
to  considering  a  symmetric  Luce  character.  Similar  analysis  can  be 
performed  with  nonsymmetric  Luce  character,  but  the  generality 
seems  to  obscure  the  nature  of  the  result.  The  exponential  charac¬ 
ter  and  corresponding  ROC  curve  are 


A+l  A'Z0 
vA  '  A- 1  2 

A  -  1  0 


7  N  Zq  =  In  A 


Z_~J0 
A  1 


_A -!o 

XE(z)  e  A''1  ~  Y^(z) 


(TO.  50) 


The  terminal  observer's  character  is  formed  bv  convolution 


s(z)  w.(za,I"e(z-za)  ‘  “e(z  +  2a) 

L 


'  4  A  r~  ■*  \ 

lo  on 


Each  of  the  exponential  terms  in  this  convolution  can  be  simplified. 
Using  the  double  notation  these  are  considered  simultaneously.  Each 
i  ••  "mi  explicitly  in  the  exponential  form  of  Eq.  10.  50.  and  a  com¬ 
mon  exponential  factored  from  each. 
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»E(z  +  zA) 


a  i  Z  "  Zn  4-  Z  » 

A+ 1  0  A 

VA  '  A- 1  2 

a -I  e 


-  0 


_  A+l  A 

+ A-l  2  ,  . 

e  te(z),  z>zq±za 


z  <C  Zq  i  zA 


(10.  52) 

Care  must  be  taken  to  keep  track  of  the  lower  limit  of  each  of  these 
*  quations,  since  they  differ.  Rewrite  the  ROC  character  as  Eq. 

10.  53. 


•j)  (z  a  ) 
7 r  A' 


A  A 
A-l 


ZA  A+l  ZA 

2  A-l  T 

+  e 


ffg(z) 


z  >  z. 


r  (z.) 
XT  A 


+  e 


A+l  ZA 
A-l  2 


7rE(z) 


Z0  ”  ZA 


z0  +  ZA 


z  •;  z0-  zA 


(10.  53) 


The  important  feature  of  this  ROC  character  is  that  in  each  of  the  two 
active  regions,  the  character  is  proportional  to  the  original  exponen¬ 
tial  character.  The  coefficients  of  proportionality  depend  on  the 
quality  of  the  Luce  observer  and  the  index  f<  r  the  exponential  observ¬ 
er.  as  do  the  ranges.  However,  they  are  constants  as  far  as  the  ROC 
character  is  concerned.  Let  us  emphasize  this  constant  nature  by 
using  the  two  symbols,  B  and  C. 
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Use  Eq.  10.  55.  and  the  power  law  relation  on  the  original  exponential 
ROC  curve,  to  obtain  a  power  law  relation  for  this  ROC  also. 


X(z)  -  B  Y^(z)  =  B1-A  YA(z)  z  >  zq  +  za  (10.  56) 


This  is  not  a  pure  power  ROC.  This  equation  only  holds  for  the  initial 
part  of  the  ROC  curve.  To  obtain  the  remainder  of  the  ROC  curve 
integrate  the  second  part  of  Eq.  10.  54.  This  is  most  simply  done  by 
integrating  in  the  direction  opposite  the  usual,  by  integrating  from 
the  lower  limit  up  to  some  value  of  z,  and  obtaining  the  probability 
of  miss  and  the  probability  of  correct  rejection. 

1  -  X(z)  =  c[l  -  XE(z  +  zA)].  l-Y(z)  *  c[l- Ye(z  +  za)] 

!Z"Z0|  <  ZA 

(10.  57) 

The  expression  for  the  original  power  law  is  used  to  simplify  (lO.  57) 
to 
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1  - 


1  -  X(z) 


1  - 


-  Y(z)1A 
C  j 


z  -  z. 


Evaluate  B  and  C  by  comparing  Eqs.  10.  53  and  10.  54. 


(10.  58) 


B  -  cosh 


/a+i  za\  /  ,/za\ 

^  A-  1  2  ]/  coshy  2  )  ’ 


B  >  1 


C  -  co  (z . )  =  .  5/cosh 
7  A  ' 


,sh(-r) 


.  c 


25 


(10.  59) 


( 10.  60) 


The  constant  B  is  always  greater  than  one,  and  is  much  greater  than 
one  vvhen  the  Luce  observer  is  far  superior  to  the  original  exponen¬ 
tial  observer.  The  portion  of  the  ROC  curve  beginning  at  the  origin 
s  a  modified  power  curve.  It  has  the  same  power  as  the  original 
exponential.  However,  the  false  alarm  rate  will  be  less.  To  obtain 
the  numerical  idea  of  how  much  less,  consider  the  power  A  •-  2.  5, 

which  is  m  the  neighborhood  of  the  normal  Vd  -  1.  For  z.  =  1.  45. 

Pi 

B  2.  22,  and  the  false  alarm  ratio  between  original  exponential  and 
final  modified  power  is  .302.  When  z^  2.95.  B  =  6.  79.  and  the 
false  alarm  factor  is  .  063. 

The  primary  purpose  of  these  examples  has  been  to 
show  how  the  multiple  observer  can  be  treated  by  using  the  ROC  char¬ 
acter.  Examples  could  be  similarly  given  to  show  how  a  completely 
discrete  set  of  subordinate  observers  could  be  quickly  convolved,  and 
the  resulting  polygon  RGC  curve  calculated.  A  second  purpose  has 
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also  been  to  show  the  role  of  a  relatively  inferior  continuous  observ¬ 
er  operating  in  conjunction  with  a  superior  discrete  observer. 

10.  4  Rating  Scale  Pay-Off  Matrices 

This  section  deals  with  experimental  techniques  used  to 
obtain  ROC  curves  from  subjects  who  can  be  motivated  by  reward  and 
punishment.  In  the  basic  procedure  for  determining  the  observer's 
ROC  curve  (the  yes- no  experiment),  a  single  decision  is  obtained 
from  the  observer  on  each  observation.  An  operating  point  is  deter¬ 
mined  by  making  a  sufficiently  large  number  of  observations  to  ob¬ 
tain  the  accuracy  desired.  In  order  to  obtain  the  shape  of  the  ROC 
curve,  the  observer  must  repeat  this  observation-decision  experi¬ 
ment,  operating  at  a  number  of  different  operating  points  along  the 
ROC  curve.  Instructions  by  the  experimenter  can  be  used  to 
motivate  the  observer  to  shift  his  decision  mechanism  to  obtain  these 
different  points.  Another  technique  that  may  be  used  is  to  establish 
a  bonus  system,  or  pay-off  matrix,  for  the  experiment.  The  subject 
is  given  a  four- entry  table  which  shows  him  exactly  how  much  he  will 
be  rewarded  for  each  correct  decision,  and  how  much  he  will  be 
fined  (or.  rewarded  Jess)  for  each  incorrect  decision.  The  observer 
also  knows  the  a  prion  probability  of  each  of  the  conditions  N  and 
SN.  He  is  assumed  to  operate  in  a  manner  which  will  tend  to  maxi- 
mi^  his  total  bonus.  The  true  motivating  utility  to  the  observer  may 
not  be  precisely  what  the  pay-off  matrix  describes,  since  he  may 
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take  into  account  many  other  factors  in  choosing  his  operating  point. 
However,  when  the  pay-off  matrix  is  changed,  almost  all  observers 
will  respond  with  a  change  in  the  operating  point  that  is  in  the  same 
general  direction  as  the  operating  point  for  an  observer  motivated 
solely  by  the  pay-off  matrix.  The  bonus  system  also  gives  a  form 
of  feedback  to  the  observer,  who  can  try  to  adjust  his  own  internal 
description  of  a  cut  on  a  decision  axis  sc  as  to  maximize  the  bonus. 
This  type  o  jdback  may  tend  to  stabilize  the  experimental  situation 
and  aid  in  the  repeatability  of  particular  operating  points. 

This  procedure  for  determining  an  ROC  curve  point  by 
point  can  Kike  considerable  time.  A  procedure  which  might  establish 
four  or  six  or  even  fifty  operating  points  on  an  ROC  curve  in  a  single 
experimental  session  is  a  tremendous  boon.  Such  a  procedure  is 
the  rating  technique.  In  the  fixed  category  rating  technique,  the  ob¬ 
server  may  make  one  of  a  fixed  number  of  responses.  One  end  of 
the  response  scale  indicates  "A"  with  a  great  deal  of  confidence,  and 
the  other  extreme  represents  ”B"  with  a  great  deal  of  confidence, 
while  intermediate  ratings  represent  intermedia!  preferences  for  the 
”A”  and  "B"  decisions.  This  experimental  technique,  a  discussion 
of  its  advantages,  and  experiments  verifying  that  it  yields  essentially 
the  same  results  as  the  longer  yes- no  procedure  are  given  by  Egan, 
Schulman,  and  Groenburg  (Ref.  34,  and  Ref.  8,  p.  172).  They 
used  a  four-category  technique.  Watson  and  Rilling  (Ref.  33)  have 
used  an  essentially  continuous  rating  technique  to  obtain  ROC  curves. 
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In  a  study  by  Swets,  Tanner  and  Birdsall  (Ref.  26 , 

■t  i  >  *  •'  •  i 

also  Ref.  8,  page  130)  a  six- category  rating  scale  was  used  to  ob¬ 
tain  an  ROC  curve.  However,  the  intention  of  that  experiment  was  to 

.  •'  t.  v  •  .  i  ■;  -r  "  .«  ‘ 

determine  more  than  just  the  ROC  curve.  The  observers  were  in- 

structed  as  to  the  meaning  of  each  category.  Specifically,  they  were 

;  .  v  .  r.«  ,• .  -v  ‘  /•.  Av  .  .  «  .  , 

informed  that  the  highest  category  should  mean  an  a  posteriori  proba¬ 
bility  of  signal  presence  of  roughly  ninety-five  percent  or  better,  that 
the  next  lower  category  must  indicate  between  eighty  and  ninety  five 
percent  a  posteriori  probability,  etc.  The  technique  was  satisfac¬ 
tory  in  obtaining  a  six  point  ROC  curve,  but  the  experimenters  were 
basically  dissatisfied  with  the  relation  of  the  responses  to  the 
assigned  a  posteriori  categories. 

In  personal  communications.  Tanner  has  argued  that 
the  pay-off  matrix,  with  its  monetary  entries,  is  a  simple,  clearly 
understood,  and  efficient  means  of  explaining  the  experimenter's 
desires  so  that  the  observer  can  understand  them.  This  is  not  a  re¬ 
flection  ,>n  the  observer's  intelligence,  but  on  his  experience.  The 
pay-off  matrix  deals  with  quantities  more  familiar  to  the  observer 

.  •  ■  <*'  ’  i  *  / 

than  such  quantities  as  a  posteriori  probability,  likelihood,  "strict" 

.  i  v  ..I;'! 

versus  "medium"  criterion,  or  "nine  percent  false  alarm  rate.  "  A 
bonus  system  also  affords  the  observer  a  specific  measure  that  he 
can  check  after  an  experimental  run,  to  see  to  what  extent  he  is  hold- 
ing  a  particular  criterion. 

\  h  . i i •--.I.  *•,  "i* is ,  *  :*>  .•  '<  1  ‘ 


358 


The  purpose  of  this  section  is  to  present  a  type  of  pay¬ 
off  matrix  to  use  ir.  conjunction  with  a  continuous  rating  scale  report 
by  the  observer.  It  also  presents  a  formula  for  tailoring  this  rating 
scale  pay-off  so  that  the  ideal  observer,  "motivated  solely  by  the 
pay-off  matrix,  "  would  report  a  rating  proportional  to  an/  one  of  a 
number  of  functions  which  the  experimenter  specifies. 

The  pay-off  matrix  for  a  standard  yes- no  experiment, 
and  the  appropriate  cut  value  on  the  likelihood  ratio  decision  axisj 
is  shown  in  Fig.  10.  5. 


Fig.  10.5.  Fixed  pay-off  and  £- cut  matrix 

The  only  restriction  on  the  values  in  the  table  is  that  the  value  of  a 
correct  response  must  be  greater  than  the  value  of  the  correspond¬ 
ing  incorrect  response.  Thus,  when  noise  alone  is  present  the  value 
of  a  correct  rejection  must  be  greater  than  the  value  of  a  false  alarm, 
when  signal  plus  noise  is  present,  the  value  of  a  detection  must  be 
greater  than  the  value  of  a  miss.  This  section  will  start  with  an  ex¬ 
ample  of  one  type  of  rating  scale  pay-off  matrix  and  then  generalize 
in  two  steps. 

The  observer  fe  reporting  procedure  will  be  as  follows. 
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He  makes  the  customary  binary  decision  "A",  ”B’\  and  reports  a 
magnitude  u.  If  he  is  correct,  he  will  be  paid  proportional  to  u.  If 


he  is  incorrect*,  he  will  bfe  penalized  proportional  to  u2.  Thus,  he 


will  wish  to  make  the  value  of  his  rating  response,  u,  large  in  order 


to  obtain  a  large  reward,  but  this  is  done  at  the  risk  of  a  very  much 

~  7?'S\ 

larger  penalty.  In  addition  to  the  description  in  terms  of  the  observ¬ 


ers  rating,  u,  a  constant  k  is  fixed  by  the  experimenter  to  scale  the 

;  •  •'>'  v'  ■ 

positive  pay-offs  (to  keep  the  experimenter  from  financial  ruin)  and 

.  ...  i  S  '  '  ' ,  i  t  T  i‘  ' 


a  factor  R  is  used  to  give  a  relative  weight  of  the  N  pay-off  to  the 
SN  pay-off. 


k  >  0  a  fixed  scale  factor  for  positive  pay-off1 
R  >0  a  f  ixed  factor  for  N  pay-off  over  SN  pay-off  (10.  61) 

u  >  0  , ,  magnitude  of  observer’s  rating  response 


v  .  '  ]  ,  *  i  t  -  *  ’  i  '  >  i  * 

Using  these  constants  the  pay-off  matrix  for  what  will  be  called  the 
C-rating  scale  is  given* in  Fig.  10.  6.  4 
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Fig.  10.6.  Pay-off  matrix- for  rating  scale 
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Consider  for  a  moment  any  fixed  value  for  u.  The  proper  cut-off 
on  the  likelihood  ratio  axis  can  be  obtained  by  using  the  pay-off  ma¬ 
trix  of  Fig.  10.  8  in  the  optimum  cut  equation  of  Fig.  10.  5. 


n  P(N)  ukR  +  .  5u2 R  P(N) 
P(SN)  '  uk  +  .  5u2 


(10.  62) 


The  cut  on  the  likelihood  ratio  axis  is  independent  of  the  value  of  the 
rating,  u.  Therefore,  before  the  observer  has  chosen  the  magnitude 
of  rating  for  a  particular  response,  he  may  determine  whether  the 
"A"  or  "E"  decision  is  appropriate. 

In  the  analysis  the  sole  motivation  of  the  observer  is  to 
maximize  his  average  pay-off.  Whenever  his  response  is  "A”,  he  will 
wish  to  choose  the  rating  value  u  to  maximize  the  expected  pay-off 
for  that  particular  single  response.  This  value  is  given  in  Eq.  10.  63. 

f  VA(f)  =  P(SN|x)  uk  -  .  5uzR  P(Nlx)  (10.63) 


The  a  posteriori  probability  appears  in  this  equation  because  the 
choice  of  the  rating  will  be  made  after  the  observation  x.  The  rat¬ 
ing  u  is  the  only  variable  in  this  equation.  The  proper  u  value  is 
obtained  by  differentiating  the  value  V.  The  first  and  second  deriva¬ 
tives  of  the  value  with  respect  to  u  are 


8VA(t> 

?u 


=  P(SNlx)  k  -  uR  P(N|x) 


(10.84) 


1 
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A—  =  -RP(Nix) 


(10.65) 


3  u* 


,  K^j.V 


,  •  ’  Y  f.: 


Since  the  second  derivative  is  everywhere  negative,  the  value  has  a 

y ,  ‘ ,  y,< yi  ■  . ..  »  •  •  . » •  ■  ■ ; 

single  maximum  at  the  zero  in  the  first  derivative.  From  Eq.  10.  64, 


the  derivative  is  zero  whenever 


k  P(SNlx)  _  k  P(SN)  _  * 
u  =  R  P(Nlx)  =  R  PTnT  W  ~  P 


k 


(10.  66) 


The  constants  P  and  R  are  related  to  the  a  priori  probability  ratio, 
and  are  used  to  eliminate  this  a  priori  probability  from  the  final  ex¬ 
pression  in  Eq.  10. 66.  The  same  type  of  analysis  is  used  to  deter¬ 
mine  the  proper  magnitude  of  the  rating  response  when  the  decisioh 
response  has  been  "B". 


£  <  ft  Vn(f)  =  P(Nlx)  ukR-  .  5u2  P(SN!x)  (10.67) 

B 


0VROO 

— =  P(Nlx)  kR  -  u  P(SNlx) 
3  u 


02vo(f) 


—  =  -P(SNlx) 


u  n  P(Nlx)  _  kR  _?(N)  1  _  M 

J  "  k R  p(sn1x)  "  k  *  p(sn5 $3  ’■  i 


J  .  J  i  '  * 


Let  us  summarize  the  effect  of  this  type  of  pay-off  ma- 


(10. 68) 


(10. 69) 


(10.70) 
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trix  on  the  observer  who  is  motivated  solely  by  maximizing  his 
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expected  return.  His  binary  decision  is  the  same  as  with  a  fixed 
pay-off  matrix  with  value  ratio  equal  to  R.  Whenever  his  response 
has  been  "A’,'  his  rating  is  directly  proportional  to  the  likelihood  ra¬ 
tio.  Whenever  his  decision  has  been  "B",  his  rating  response  is 
inversely  proportional  to  the  likelihood  ratio.  The  constants  of  pro¬ 
portionality  are  established  by  the  experimenter. 

What  can  be  learned  from  such  an  experimental  tech¬ 
nique?  As  long  as  the  observer's  rating  response  is  order-preserv¬ 
ing  with  likelihood  ratio,  a  valid  ROC  curve  can  be  obtained  with  this 
method  as  with  any  rating  scale  method.  If  enough  observations  taken 
under  suitably  stable  conditions  have  been  used  to  obtain  this  ROC 
curve,  an  analytic  fit  can  be  used  to  approximate  the  curve.  From 
the  analytic  fit  the  distribution  function  can  be  obtained  for  the  likeli¬ 
hood  ratio,  as  in  any  of  the  analytic  models  in  this  present  work. 

These  distributions  can  be  compared  with  the  distributions  of  the 
observer's  u  values.  The  experimenter  can  thus  obtain  informa¬ 
tion  about  the  degree  to  which  the  observer  has  been  able  to  match 
his  u-jcale  with  the  £-scale. 

The  first  step  in  generalization  of  this  type  of  pay-off 
matrix  is  to  determine  if  an  ideal  observer  can  be  forced  to  report  a 
rating  of  any  other  nature  than  one  proportional  to  lixelihood  ratio. 

In  this  generalization  it  may  be  necessary  to  restrict  the  range  of  the 
observer's  available  rating  responses.  A  function  r(u),  a  rating 
forcing  function,  will  play  the  same  role  as  u2  in  the  f- rating  pay-off 


363 


matrix. 

u  >  Uq  magnitude  of  observer’s  rating  response 

R  >  0  a  fixed  factor  for  N  pay-off  over  SN  pay-off 
r(u)  >0  a  smooth  convex  upward  function  for  negative  pay-offs 

(10.71) 

The  four- fold  pay-off  matrix  is  shewn  in  Fig.  10.  7. 


Fig.  10.  7.  Base  pay-off  matrix  for  general  rating  scale 

At  this  stage  in  the  development  the  rating  forcing 
function  r(u)  will  be  given  some  additional  properties  to  make  the 
solution  unique.  After  this  has  been  completed  and  a  single  solution 
obtained  for  each  situation,  the  rating  forcing  function  may  be  gener¬ 
alized  to  make  it  more  appropriate  for  particular  practical  situations. 
These  restrictions  are  given  in  Eq.  10.  72. 

r0(uQ)  =  uQ  .  t^Uq)  =  1  ,  rJJ(u)  >  0  ,  u  >  uQ  (10.  72) 

The  first  restriction  is  used  so  that  the  pay-off  matrix  will  be 
’’balanced”  when  the  observer  chooses  the  minimum  rating  value. 
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That  is.  the  value  for  detection  will  be  as  much  positive  as  the  value 
for  miss  is  negative,  and  the  value  for  correct  rejection  as  much 
positive  as  the  value  for  false  alarm  will  be  negative.  The  second 
restriction  will  insure  mat  a  solution  exists.  The  third  will  always 
hold,  the  function  must  be  a  convex  upper  function;  this  is  the  char¬ 
acteristic  that  restrains  the  observer  from  using  extremely  large 
scale  values. 

The  first  step  is  to  determine  a  proper  operating  point 
by  using  the  /3  equation  of  Fig.  10.  5  for  the  pay-off  matrix  of  Fig. 
10.7. 


R  P(N)  R  u  +  R  r0<u>  _  p(N)  p 
'  Pfl3N)  u  +  r0(u)  pSnJ  K 


(10.73) 


As  expected,  the  observer  may  determine  the  proper  response  thresh¬ 
old  to  cut  between  the  "A”  and  "B"  decisions.  As  before,  the  deriva¬ 
tive  of  the  value  function  conditional  to  the  binary  decision  response 
yields  the  appropriate  u  value.  The  second,  derivatives  are  indeed 
negative  and  are  not  displayed  here.  The  values’  first  derivatives 
and  the  equation  controlling  the  optimum  rating  response  are  given 


in  Eqs.  10.  74  - 

10.  79. 

l  >  P 

VA(Jt)  =  P(SNlx)  u  -  R  rQ(u)  P(Nlx) 

(10.  74) 

=  P(SNIx)  -  R  Tq(u)  P(Nlx) 

(10.75) 
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r0(uA> 

vB«) 

3VB(t) 

?u 

r0(uB> 


1  P(SNlx) 

=  r  =  y 

=  P(Nlx)  R  u-  rQ(u)  P(SNlx) 


=  F(Ntx)  R  -  r£(u)  P(SNix) 


»  P(Nlx) 
R  P(SNlx) 


(10.  76) 

(10.77) 

(10.78) 

(10.  79) 


Equations  10. 76  and  10.  79  appear  to  repeat  virtually 
the  same  situation  as  (10.  66)  and  (10.  70),  namely,  that  some  function 
of  the  rating  response  be  directly  proportional  to  the  Si  over  j3  ratio 
under  condition  A  and  inversely  proportional  to  the  same  ratio  under 
condition  B.  This  is  the  basic  restriction  on  this  present  technique. 
The  rating  may  be  forced  to  follow  some  particular  function  of  the 
experimenter's  choice  only  if  the  experimenter’s  function  is  in  turn 
a  function  of  the  £  to  *3  ratio.  However,  in  the  study  of  decision 
processes  based  on  likelihood  ratio  there  are  a  number  of  functions 
of  this  type  of  interest. 

One  more  step  of  generalization  is  called  for  before 
displaying  particular  cases  of  rating  scale  forcing  functions.  The 
pay-off  matrix  of  Fig.  10.  7  can  be  changed  by  the  addition  of  a  prop¬ 
erly  selected  set  of  four  numbers  which  will  change  the  pay-off  to 
the  observer  but  will  not  change  any  of  the  equations.  That  is,  the 
decisions  and  the  rating  scale  reports  will  be  exactly  the  same,  but 
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the  monetary  return  to  the  observer  will  be  different.  Assume  that 
the  experimenter  has  been  successful  in  solving  Eqs,  10.  76  and  10.  79 
for  his  particular  rating  function.  Once  he  knows  the  value  of  the 
minimum  rating,  Uq,  he  may  choose  any  three  constants,  Kq,  Kp 
and  Kg,  so  long  as  Kg  is  less  than  2uq. 

Kq,  Ki,  arbitrary,  Kg  <  2uQ  (10.30) 


The  role  of  these  three  arbitrary  constants  is  shown  in  Fig.  10.  8. 


R:\c:  SN  N 
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Kj  +  u 

Kq  +  R  Kg  -  R  rQ(u) 

K1  +  K2  ‘  r0(u) 

*0**. 

Fig.  10.  8.  Formation  of  complete  pay-off  matrix 
from  base  matrix 


The  resultant  pay-off  matrix  must  be  a  valid  pay-off  matrix,  and  not 
affect  the  optimization  equations.  To  be  a  valid  pay-off  matrix,  the 
value  paid  for  detection  must  be  greater  than  the  value  paid  for  a  miss. 

VD  >  Ym:  +  u  >  -  rQ(u)  (10.  81) 

The  constant  appears  on  both  sides  of  the  equation  and  therefore 


« 


disappears.  This  leaves  the  restriction 

Tq(u)  +  u  >  Kg  (10.  82) 

Because  the  forcing  function  Tq(u)  is  monotone  increasing,  it 
takes  on  its  smallest  value  at  Uq. 

rQ(u)  u  >  r0(u0)  +  uQ  =  2uQ  (10.  83) 

By  definition  Kg  is  smaller  than  this  value,  and  the  value  of  detec¬ 
tion  is  indeed  greater  than  the  value  of  a  miss.  Similarly  the  value 
for  a  correct  rejection  is  greater  than  the  value  paid  for  false  alarm. 

VCR  VFA:  Kq  +  R  u  >  Kq  +  R  Kg  -  R  rQ(u)  (10.  84) 


Subtract  Kq  from  both  sides  of  this  equation,  divide  by  R,  and  ob¬ 
tain  Eq.  10.  82  again.  The  final  pay-off  matrix  in  the  /3  equation  is 


P(Nj  K0  +  Ru 

K0  +  R  Kg  -  R  rQ(u)] 

^  Kjiu  - 

[K1  +  K2-r0(u) 

*•  — 

(10.  85) 


t  = 


P(N) 

P(SNj 


R  u  -  R  Kg  +  R  Tq(u) 


u  '  K2  +  V 


P(N)  R 

pm  R 


(10.  86) 


the  same  (3  equation  as  always.  Although  Kq  and  Kj  disappeared 
from  the  (3  equation,  they  will  appear  in  the  value  equation  under  the 


conditions  A  and  B. 
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t  >  f3  VA(i)  =  P(SNlx)  [Kx+  u]  +  P(Nlx) 

.[K0  +  HK2-  Rr0(u)]  (10.87) 

However,  when  one  differentiates  to  find  the  maximum,  these  constants 
disappear  and  the  same  equation  as  before  appears. 

dVjt) 

=  p(SN,x)  “  R  *&<">  P(Nlx)  (10.  74) 

Similarly,  the  constants  appear  in  the  equation 

(  <  0  V B(l)  =  P(Nlx)  [Kq  +  Ru]  +  P(SNlx) 

•[K0  + RK2  -  RrQ(u)]  (10.88) 

but  disappear  upon  differentiation. 

a  vR(f) 

— =  P(N|x)  R  -  Tq(u)  P(SN|x)  (10.77) 

The  experimenter  therefore  has  a  great  deal  of  freedom  in  choosing 
the  actual  pay-off  matrix  that  he  will  use  in  his  practical  experimen¬ 
tal  situation.  To  determine  the  rating  scale  forcing  function  we  may 
consider  the  simpler  base  pay-off  matrix  of  Fig.  10. 7. 

In  the  remainder  of  this  section,  three  rating  scale 


forcing  functions  will  be  developed,  both  for  their  own  sake  and  to 
demonstrate  the  technique  for  determining  such  a  forcing  function. 


Example: 


Desire  =  M'jg)  and  u 


B 


The  basic  tools  are  Eqs.  10. 76  and  10.  79.  These  are  symmetric 

equations.  Equation  10.  76  shall  be  solved  first  in  each  case,  and 

then  Eq.  10. 79  checked  to  see  if  it  is  satisfied.  Equation  10.  76  says 

that  the  first  derivative  of  r  evaluated  at  the  proper  value  of  u 

will  be  equal  to  the  £  to  /3  ratio.  The  desired  result  is  to  have  the 

k£ 

proper  value  of  u  equal  to  -j-  .  This  requires  that  r'  evaluated 

k£  £ 

at  -jjj-  equals  ^  • 

ro(k|!  *  |  =*  ro(u)  =  E 

This  determines  the  equation  for  the  derivative  of  r.  The  side  con¬ 
ditions  of  Eq.  10.  72  set  the  value  of  r’  at  the  minimum.  Since  the 
minimum  of  the  £  to  /3  ratio  is  one,  Uq  is  equal  to  k. 

r0(u0>  =  1  =*  u0  =  00 


The  function  Tq(u)  is  obtained  by  integration  of  Tq(u).  Integration 
introduces  an  arbitrary  additive  constant,  normally  suppressed  but 
displayed  here 

rQ(u)  =  /  r()(u)  du  -  /  £  du  =  -2k  +  c 
The  first  side  condition  of  Eq.  10. 72  determines  this  arbitrary  con¬ 


stant. 
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r0(u0)  =  “0  =*  C  =  1 

Gathering  these  all  together,  the  final  form  of  the  basic  rating  forc¬ 
ing  function  which  will  make  the  response  proportional  to  the  likeli¬ 
hood  ratio  under  the  A  condition  is 


Although  not  displayed  1.  re,  this  does  check  with  Eq.  10,  79.  This 
function  differs  somewhat  from  that  given  in  the  original  example. 

The  difference  is  that  the  original  example  hud  a  constant  added  to 
Tq(u)  to  form  the  simpler  appearing  function 

ro(u)  '  T  “  fir  =  r(u) 

This  forcing  function  is  displayed  in  Fig.  10.9(a). 

Example:  Desire  u  =  klz  -  In  0) 

Writing  z  as  In  f,  we  insert  the  desired  form  of  u  in  the  equation 
for  the  derivative  of  r. 

A:  rj(k(ln  £  -  In  $))  =  |  =*  rj(u)  = 

The  side  condition  on  the  derivative  of  r  at  the  minimum  value  deter 
mines  that  the  minimum  value  la  *ero. 


K  P(SNlx) 


K.  P(Nlx) 


Fig.  10.9.  Rating  forcing  functions 


372 


ri(uQ)  =  1  =>  uQ  =  0 
Integrating  the  derivative  to  obtain  the  function 

rQ(u)  =  k  eu^k  +  c 

and  using  the  side  condition  to  determine  the  constant  of  integration 

r0(0)  =  0  *  C  =  -k 

the  final  form  of  the  forcing  function  is  determined. 

r0(u)  =  k  [eu/k  -  lj  u  >  O' 

This  forcing  function  is  shown  in  Fig.  10.  9(b), 

ExamEie:  Desire  «A  =  k  P(SNIx)  and  uB  =  k  P(Nlx) 

The  goal  is  to  choose  the  forcing  function,  rQ(u),  so  that  the  observer 
will  report  u  values  that  are  proportional  to  a  posteriori  probabil¬ 
ity,  P(SNlx).  The  formal  solution  for  the  ”A"  condition  has  a  solu¬ 
tion,  but  it  differs  from  the  HB"  solution  unless  R  and  P{SN)/P(N) 
are  both  one. 

Restriction:  R  =  1,  p(SN)  =  .  5  =  P(n) 


If  this  restriction  is  made,  the  solution  is 
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rn(u)  =  k  jl  -  In  2  -  £  -  ln(l  -  -jj) 


f  <  u  <  k 


D9Ui 
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.(y  ~  i  -  i)  Hi  oa  /I9iTj  ,o'4j;rn;  onj  ni  a> 

mine  a  specific  ROC  curve.  Those  in  psychophysics  will  find  a 

.  ,o<r;  t 

variety  of  curves  and  characters  that  may  be  more  like  their 

sdi  no  "  iximun  Inoi  y« /;  ‘Kit  :iun*  ^.ib  ‘otnuro  on  1  T 

experimental  data  than  the  previouslv  available  ROC,  curves. 

aoiamb  bust  moons  1  on?  louteno-j  .  'SViylm  linu  Doao.u 

Every  result  obtained  suggests  further,  research.  Especially 

lnr.lanoo  yrn;  lo  -ju n;v  boiooqxe  *yp  .1  e  y 

needed  is  research  in  fitting  ROC  curves  ^nd  cl^acljers)  to  experi¬ 
mental  data  so  that  one  may  ektra^^ivhkt^nf^r^tion  is  truly  present, 
by  placing  appropriate  weighting  on  each  datum.  work  has  no 
conclusion,  only  a  continuat^  iiv  t^e  (j^logn^wit  of  our  understanding 

of  the  evaluation  of  decision  devices.  . .  r.  ~  ,0x 
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APPENDIX  A 


TWO  THEOREMS  ON  ROC  CURVES  AND  LIKELIHOOD  RATIO 

This  appendix  is  part  of  the  notes  for  one  lecture  issued 
to  the  students  enrolled  in  ’’Random  Processes,”  a  University  of  Mich¬ 
igan  Engineering  Summer  Conference,  in  the  years  1959  through  1963. 

It  is  included  in  this  present  work  because  it  contains 

the  proofs  of  theorems  herein  called  one  and  two.  A  more  general  proof 
of  one-half  of  the  second  theorem  is  given  in  Chapter  I. 

Theorem  1:  Given  two  simple  hypotheses,  the  ROC  image  of  all  ran¬ 
domized  decisions  is  convex,  and  contains  the  chance  diagonal  P(AlSN)  = 
P(AiN).  It  is  symmetric  through  the  center  (.  5,  .  5),  that  is,  if  (x,  y) 
is  in  the  image,  then  so  is  (1  -  x,  1  -  y). 

Proof: 

(1)  The  chance  diagonal:  for  any  real  number  r  on  the 
closed  unit  interval,  consider  the  randomized  decision 
probability  g(x)  -  r.  The  expected  value  of  any  constant 
is  that  constant. 

P(" A” IN)  =  E(rlN)»  r 
and 

P("A”  ISN)  =  E(riSN)=  r 

(2)  Convex:  Consider  any  two  distinct  randomized  decisions, 
say,  g|(x)  and  ggfc).  The  ROC  images  are  PCAj”  IN), 
PC  A  ISN).  We  wish  to  show  that  lor  any  real  number  a 
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Proof  (Cont. ) 

on  the  closed  unit  interval  there  is  a  randomized  deci¬ 
sion  probability  gg(x)  whose  image  is 

[aPC'Aj"  iN)  +  (1  -  a)  P("A2"lN), 

aPC'Aj"  (SN)  +  (1  -  a)  PC’A^’ISN)] 

The  obvious  choice  is  to  try 

g3(x)  =  a  gj(x)  +  (1  -  a)  g2(x) 

Since  gj  and  g2  range  from  0  to  1,  as  does  a,  g3  will 
also;  the  sum  of  integrable  functions  is  integrable. 
Therefore  the  above  g3(x)  is  admissible  as  a  randomized 
decision  probability,  and  the  linearity  of  integration 
assures  that  the  image  of  g3(x)  is  the  desired  point  (on 
a  straight  line  between  the  images  of  gj  and  g2). 

(3)  Symmetry:  If  the  image  of  some  gj(x)  is  the  ROC  point 
(x,  y),  then  consider  g2(x)  =  1  -  gj(x).  This  has  the 
range  0  to  1,  is  integrable,  and  has  the  desired  image 
at  (1  -  x,  1  -  y).  With  respect  to  the  ROC,  g2  is  called 
"the  opposite  of  gj.” 

Q.  E.  D. 

Theorem  2:  The  collection  of  all  criteria  based  on  likelihood  ratio 
maps  onto  the  upper  boundary  of  the  ROC  region. 

Proof: 

The  two  steps  in  this  proof  are  the  "into"  and  "onto  all" 
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parts,  i.  e. ,  to  show  that  any  criterion  based  on  likeli¬ 
hood  ratio  maps  into  some  point  on  the  upper  boundary, 
and  then  that  any  point  on  the  upper  boundary  is  the 
image  of  some  criterion  based  on  likelihood  ratio. 

(1)  Consider  any  decision  based  on  likelihood  ratio 

g(x;  p,  r)  and  any  other  randomized  decision  probability, 
h(x),  with  not  greater  expected  value  under  N,  i.  e. , 

E(h(x)!N)<  E(g(x;/3,r)|N) 

Whenever 

f(xlSN)  -  pf (x ! N)  >  0,  f(x)  >  /3,  g(x;  /3,  r)  =  1 
and 


g(x;  0,  r)  -  h(x)  >  0; 
whenever 


f(xlSN)  -  f(x  i  N)  <  0,  f(x)  <  ft  g(x;ft  r)  =  0 
and 


g(x;  ft  r)  -  h(x)  <  0 

That  is 


(f(xlSN)  -  0f(xjN)]  (g(x;  ft  r)  -  h(x)]  >  0 
Therefore 

J  (f(xlSN)  -  jSf(x!N)J  [g  (x;  ft  r)  -  Kx)]dx  >  0 
Expanding 


- 


._s.  .  -  ...  .  .  „  ....  ..  . 
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E(g(x;  ft  r)  1 SN)  -  E(h(x)ISN)  -  /3[E(g(x;  ft  r)!N) 

-  E(h(x)  i  N)]  >  0 

Since  the  last  two  expected  values  have  a  positive  dif¬ 
ference  by  original  assumption  and  (3  >  0, 

E(g(x;  ft  r)  I  SN)  >  E(h(x)lSN) 

Thus  one  concludes  that,  given  any  criterion  based  on 
likelihood  ratio,  its  P("A"ISN)  value  will  be  greater  than 
or  equal  to  the  P("AM|SN)  value  for  any  randomized  deci¬ 
sion  device  with  equal  to  smaller  P("AM !  N)  value.  Con¬ 
sidering  only  equal  values  of  P(”A"!N)  is  all  that  is 
needed  to  show  that  any  criterion  based  on  likelihood 
ratio  maps  into  the  upper  boundary  of  the  ROC,  concluding 
step  one. 

(2)  One  wishes  first  to  show  that  for  any  k,  0  <  k  <  1,  that 
there  is  a  nonnegative  (3^  such  that 

E(g(x;  0)lN)<  k  <  E(g(x;  ^  1)  I N) 
since  if  one  of  the  equalities  holds,  that  strict-criterion 
will  have  the  appropriate  P(AlN)  value,  and  if  neither 
equality  holds,  the  value 

k  -  E(g(x;  0)|N) 

rk=  E(g(x;  l)|N)  -  E(g(x;  ^  0)IN) 


Proof  (Cont. ) 

will  yield  the  appropriate  probability 
E(g(x;  Pk>  rR)|  N)  =  k 

Although  it  may  be  obvious  to  many  that  E(g(x,  p,  1)|N) 
is  continuous  on  the  left  as  a  function  of  p ,  and  that 
E(g(x;  p,  0)lN)  is  continuous  on  the  right,  and  that  these 
functions  having  common  points  of  continuity  and  discon¬ 
tinuity  by  their  very  definitions  establish  the  proof, 
the  following  simple  steps  may  help  others  to  that  con¬ 
clusion. 

At  the  extreme  left  of  p  =  0,  E(g(x;  0,  1)  I N)  =  P(/?  >  0[N)  =  1 
Toward  the  right  extreme  of  arbitrarily  large  p  note  that 

E(g(x;  P,  r) I SN)  =  f  g(x;  p,  r)  f(x)f(xlN)dx  >  pfg(x;  p,  r)  f(x(N)dx 


that  is 


E(g(x;  ft  r)lN)  <  j  E(g(x;  ft  r)ISN)  <  | 


so  that 


lim  ,  \ixt\  -  lim  1 

3-KE<f!<x;“.r)IN)b  5  =0 

For  any  point  p^  not  zero  or  infinity,  if  p  <  p^ 

P(f(x)  >  ,31N)  =  P(Kx)  >  pQ  |N)  +  P (pi  Six)  <  pj IN) 
Since  as  ft  approaches  p^  ,  it  exceeds  all  values  less 
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“_maW<<W<  %IN)=0 

and  so 
lim 

0-0o  E(g(x;  ft  1)1  N)  =  E(g(x;  /3Q,  1) I N) 

«  ft  >  $0 

P(i(x)  >  /J|N)  +  P(/3  >  jt(x)  >  (30lN)  =  P((  >  /J0|N) 

Since  as  0  approaches  /3q  it  excludes  all  values  greater 
than 

llm  P(jf(x)>0)|N)  =  P(|>  ^ IN) 

/3“/30 

and  so 

lim 

0-Po  E(g(x;  8,  1)IN)  =  E(g(x;  pQ,  0)1  N) 

Thus  the  monotone  decreasing  (nonincreasing)  function 
E(g(x;  1)1  N)  has  extreme  limiting  values  of  1  and  0, 
and  at  any  0  q  between  zero  and  infinity  has  a  left-hand 
*  limit  of  E(g(x;  0Q,  1)1N)  and  a  right-hand  limit  of 
E(e(x;  0g,  0)lN). 

The  extreme  value  of  1  corresponds  to  the  strict  cri¬ 
terion  that  is  the  whole  observation  space  (which  is  based 
on  likelihood  ratio  in  a  trivial  sense),  and  the  extreme  value 
of  0  corresponds  to  the  strict  criterion  of  all  observations 
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with  infinite  Jikelihood  ratio  (f(xlN)  =  0  and  f(xISN)  £  0). 
The  extreme  limiting  values  serve  one  other  purpose;  for 
any  value  of  k  between  but  not  equal  to  0  or  1,  there 
will  be  a  set  of  /3  yielding  E(g(x;  /3,  1) IN)  values  be¬ 
tween  k  and  0,  and  a  set  of/3's  yielding  E(g(x;  0,  1)IN) 
values  between  k  and  1.  If  the  value  k  is  not  taken  on 
by  the  function,  then  these  two  sets  of  0’s  are  disjoint, 
and  the  cut  value  is  such  that  the  left-hand  limit  is 
greater  than  k,  and  the  right-hand  limit  is  less  than  or 
equal  to  k.  This  is  the  appropriate  and  the  value  of 
r,  was  given  at  the  beginning  of  step  2. 
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A  LOWER  TRUNCATED  SIMPLE  EXPONENTIAL  IS 


LOWER  METASTATIC  INVARIANT 


In  Section  3.  4.  2  the  statement  was  made  that  the  meta¬ 


static  image  of  a  pure  power  ROC  curve  would  be  the  same  as  the  orig¬ 
inal  if  the  point  (0, 0)  was  included  in  the  transformation.  This  state¬ 


ment  is  proved  in  this  appendix. 


Denote  the  original  ROC  as 


xi  ■  V 


A  >  1 


and  its  character  is 


vx(z)  =  (A  -  1)  A 


1  A+l  z 
,-l  A‘ A-l  "  A-l  2 


-z>  z0  = 


-In  A 


The  equations  for  the  coordinates,  as  functions  of  z,  are 

a  *-*b 


Xj(z)  =  e 


tt-t: 


Yj(z)  =  e 


XT' 


Choose  any  cut,  zQ,  greater  than  zQ.  The  metastatic  transformation 
vsill  be  to  use  only  that  part  of  ffj(z)  for  which  z  is  greater  than  zc> 


Let  a  =  0 


Yj(a)  =  0 


Vz0 


z  -z_ 
c  0 


Let  b  =  e 


Yj(b)  =  e 


X!- 


a®. 


#  K" 

iiL  '  -  . 


f  * 


The  second  ROC  will  be  the  image  of  the  rectangle. 


0  <  Xj  <  b 


0  <  Yj  <  Yj(b) 


The  shift  in  the  the  z-axis  is  given  by  Eq.  3.  39, 


z2  =  zi  +  ln  Y^bPTr^ET  zi- 


Z.,  >  Z 


z  -z_ 
.  c  0 


z  -zA 
c  0 


,  V  "a^t  /  inr 

z2  =  zi  +  ln\e  /  e 


W!z  -a0) 


z,  >  z 
1  -  c 


The  minimum  z_  value  occurs  for  z,  at  the  cut  value  z  . 

2  1  c 


rain  z2  -  zc  -  (zc  -  zQ)  =  z0 
The  new  ROC  character  is  given  by  applying  Eq.  3.  34, 


Vz2'  V  (b  -  aHYIb)  -  Y(a)) 


*1<*2  +  zc  ‘  z0> 


z  -z. 
c  0 


v*oT5 


•z2-z0 


e-^TJ 

A+l  Zc'Z0  1  A+l  z2+zc"z0 

2  (A  -  I)'1  /  **  A-_1  2 


1  A+l 

(A  -  X)'5  A  A_l  e  A_1  2 


'2(z2>  =  'l'V' 


Z2  —  *0 


* 
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APPROXIMATION  TO  A  CASE  H  CHARACTER 

The  purpose  of  this  appendix  is  to  obtain  Eq.  4. 15,  an 
almost  normal  appearing  cliaracter  to  approximate  the  a  =  2  Case  II 
character. 

The  equations  for  the  ROC  character  for  Case  II  are 
written  in  terms  of  a  parameter  of  quality,  a,  and  a  dummy,  t.  The 


tabulated  function 

•yx)  =  e'x  yx>  (4.u) 

is  used  in  computation.  The  computational  equations  are 

z-  [lnT^otJ  +  otJj  -.5a2  (4.13) 

In  v(z)  =  [.  5at  +  ln(ot)  +  1.  5  In  TQ(at)  -  lnT^at)  Jg 

-  .  5t2  -[.25tf  +  21na]  (4.14) 

Let 

B^(at)  *  In  TQ(at)  +  at  (C.  1) 

Bg(at)  =  .  5at  +  ln(at)  +  1.  51nTQ(at)  -  In  T^at)  (C.  2) 


These  were  calculated  over  the  range  of  at  from  zero  to  twenty.  The 
results  appear  in  Table  C.  1,  and  Bg  is  plotted  against  Bj  with  integer 
at  points  indicated  in  Fig.  C.  1. 

t 

These  equations  were  used  to  calculate  the  log  of  the 
ROC  character  for  a  *  2.  This  is  tabulated  in  the  fourth  and  fifth 
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columns  of  Table  C.  1,  and  plotted  in  Fig.  C.  2  for  the  z  range  of  -2 
to  +5.  This  range  of  z  corresponds  to  the  range  of  false  alarm  prob- 

i 

abilities  from  x  =  .  00004  to  x  =  1. 00. 

1 

This  arc  appears  so  parabolic  that  the  calculations 
were  continued  to  obtain  a  power-law  fit  to  the  arc.  From  Table  C.  1, 
columns  four  and  five,  it  was  estimated  that  the  mode  of  In  n(z)  occurred 
at  z  =  -.725  at  which  point  the  ln#(z)  value  was  -1.  385.  If  the  character 
were  truly  parabolic,  then  it  should  obey  the  law 

In  tt(z)  =  -1.  385  -  A(z  +  .72 5)2  (C.  3) 


If  so,  then 

-  (In  tt(z)  +  1.  385)  =  A(z  +  725)2  (C.  4) 

A  plot  of  -  (In  ff(z)  +  1.  385)  against  Iz  +  .  725!  should  plot  as  a 
straight  line  of  slope  2  on  log-log  paper.  These  last  two  quantities 
were  calculated;  they  are  listed  in  columns  seven  and  eight  of  Table 
C„  1  and  plotted  in  Fig.  C.  3. 

The  plotable  ranges  for  Fig.  C.  2  are  -2.  to  -.  887 

> 

r 

for  z  below  the  mode  of  z  =  -.725,  and  -.415  to  15.  59  for  z  above  the 
mode.  Slight  differences  in  the  fit  near  the  mode  are  inconsequential. 
The  character  above  the  mode  is  the  more  important  arc,  because  it 
covers  the  false  alarm  range  below  x  =  .  325.  All  plotted  points  above 
the  mode  are  well  fit  by  a  single  straight  line.  However,  the  slope  is 
not  two.  The  end  points  of  this  straight  line  are  given  in  lUble  C.  2. 


Graphic  determination  of  a  power  law  relation 


z  +  .725 


-(In  n(z)  +  1.  385) 

0. 22 
39.  207 


x(z) 

0.  325 

1.  93x  lO  22 
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DETAILS  OF  AN  APPROXIMATION  TECHNIQUE  FOR  SECTION  4.  3 

This  appendix  contains  a  demonstration  of  how  the  ROC 
character  can  be  used  to  obtain  an  approximation  for  the  "Case  3: 

Noise  in  Noise”  ROC  curve.  The  special  restrictions  are  that  WT  is 
large,  and  the  signal -to-noise  ratio,  S/  N,  is  small. 

Some  idea  of  the  magnitude  of  the  various  terms  involved 
in  the  ROC  character  must  be  obtained.  The  first  of  these  is  the  lower 
bound,  Zq.  The  first  several  terms  of  the  standard  approximation  for 
the  logarithm  yield 

*0  “  'WT  ln(l  +  §)B  -Wt|  (1  -  Jjj  +  -Jr> 

The  lower  bound  z^  is  negative,  and  its  magnitude  is  the  product  of  the 
large  quantity  WT  times  a  small  quantity  S/N.  For  detectability  to  be 
reasonably  good  the  range  of  the  nonzero  part  of  the  ROC  character 
must  extend  to  large  values  of  z.  Although  large  positive  values  might 
appear  to  be  sufficient  to  have  a  good  ROC  curve  for  low  values  of 
false  alarm,  one  cannot  have  large  positive  values  for  the  ROC  char¬ 
acter  without  having  large  negative  values  also.  (The  expected  value 


of  z  is  negative  under  the  condition  N. )  Make  the  further  assumption 
that  Zq  has  large  magnitude.  The  logarithm  of  the  ROC  character 


and  its  derivative 


A 

dz 


iflff(z)  = 


WT  -  1 


z  -  z, 


0 


,  N  cs  /  N 
"  ( S  +  •  5)  = 


(D.  3) 


are  going  to  be  the  tools  in  determining  the  nature  of  approximation  to 

make  for  this  ROC  curve.  The  quantity  z  in  Eq.  D.  3  is  the  mode  of 

m 

the  ROC  character. 


*  z„  +  (WT  -  1)  (-| )  (1  - 


S 

2N 


(D.4) 


Since  zA  is  roughly  -WT  S/  N  and  the  mode  z  is 
u  m 

(WT  -  1)  S/N  above  it,  zm  must  be  in  the  neighborhood  of  S/N,  that 
is,  very  close  to  zero.  Equation  D.  3,  with  mode  set  at  zero  is  very 
similar  to  a  normal  ROC  character  if  the  denominator  changes  very 
little.  To  formalize  this,  expand  Eq.  D.  3  is  a  power  series  in  z  about 
the  mode,  z  . 

7  m 


For  z  very  close  to  zm,  neglect  all  except  the  first  term 

ml! 

m“z0 


£lo»<z)*(f  +  .5)  \ 


(D.  6) 


Since  S  <<  N 
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(D.7) 

(D.8) 

(D.9) 


For  a  normal  ROC  character 


~  ln7TN(z)  =  -z/d  (D.  10) 

Can  we  treat  the  small  S/N  case  as  a  normal  ROC?  The  matching 
index  is 

d  =  (WT  -  1)  (|)2  (D.  11) 

To  answer  this  we  must  determine  how  well  the  entire  ROC  matches; 

specifically,  how  the  values  at  the  mode  match  for  the  same  index. 

Calculate  lmr(z  ). 

m 

ln7r(zm)  =  (WT  -  l)ln(zm  -  zQ)  -(  f  +  .  5)  (zm  -  zQ)  +  WTln(f  ^lT^  InrfVT) 

(D.2) 

(WT  -Dln(zm  -  z0)  =  (WT  -  l)ln  [(WT  -  1)-|  (l  +  ^ 

(a  12) 

=  (WT  -  1)  ln(WT  -  1)  +  (WT  -  1)  [n  |  -  In  (1  +  arj 
Now  (WT  -  1)  ln(WT  -  1)  is  closely  related  to  In  r.  Specifically, 
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Eq.  6.  1.  40  of  Ref.  6  gives 

In  r(A )  ~  (A  -  .  5)ln  A  -  A  +  lnVIF  +  . . . 

hence  (D* 13) 

(WT-  1)  ln(WT  -  1)  =  .  5  ln(WT  -  1)  +  (WT  -  1.  5)ln(WT  -  1) 

«.  5  ln(WT  -  1)  +  lnr(WT  -  1)  +  (WT  -  1)  -  ln/IF 


(D.  14) 

Collecting 

(WT  -  1)  ln(z  -  z  J  « .  5  ln(WT  -  1)  +  In  T(WT  -  1)  +  (WT  -  1)  -  I1W2? 
m  u 

+  (WT  -  1)  •  L  |  -  In  (l  +  (D.  15) 

Next 

'  (I*  +  •  5)  <zm  ‘  V  =  '(WT  '  l)  <D- 16) 

Hence 


ln7r(z  )  a  .  5  ln(WT  -  1)  +  in  T (WT  -  1)  -  In  T(WT)  -  ln/2?  +  (WT  -  1) 
m 


[>4  -  1“  (l  +  H5 )]  +  WT  [-|n  I  +  • 51n  (l  +  |)]  <D- 17> 

Now 

r(WT)  =  (WT  -  l)r(WT  -  1)  (D.  18) 

So 

-In  r(WT)  =  -ln(WT  -  1)  -  lnr(WT  -  1)  (D.  19) 
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lnir(im)*-.  5(WT-l)-ln/2F  -In  (|)  '  (WT  -  ])ln  (l  +  J^+.bWTln^l  + 


(D.  20) 


The  first  three  terms  can  be  written  as 
-In  i 


2tt(WT  -  1)  (|)  =  -In  V2tt  d 


(D.  21) 


Expand  the  last  two  terms  in  power  series 


-(WT  -  1)  In 


(1  +  A)  ■' 


(WT-  1) 


S  1  S2 


1 

+  — 


+  .  5WTln 


('•9- 


5  WT 


2N  2  4N2  3  8N3 


S  1  jf  1  S3 
K  “  2  +  3  ‘  n» 


(D.  22) 


(D.  23) 


Adding,  and  collecting  in  powers  of  S/  N 


s 

2N 

|  -(WT  -  1)  +  WT] 

ii 

S2 

8N2 

[  (WT  >  1)  -  2WT  ] 

=  -(WT  +  1)  — ~ 
8N* 

S3 

24N3 

l  -(WT  -  1)  +  4WT] 

=  (3WT  +  1) 

24N* 

S4 

64N4 

( (WT  -  1)  -  SWT  ] 

=  -(7WT  +  1)  — - 
64N 

(D.  24) 

Carefully  review  the  assumed  order  of  magnitude  of  the  variables. 

(1)  S/N  is  quite  small,  S/N  <<  1 


^00 
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If  d  is  to  be  the  detectability  index,  it  should  range  from  .  5  to  50. 

\ 

Hence 

(2)  (WT  -  1)  (S/N)2  is  between  .  5  and  50 
(1)  and  (2)  mean 

(3)  (WT  -  1)  S/  N  is  very  large 


Putting  these  all  together  means  the  second  term  in  the  above  collec¬ 
tion  from  the  two  power  series  is  outstandingly  large  compared  to  the 
rest.  It  is 


-{WT  +  1) 


OKI® 


=  -(WT  -  1) 


8N2 


S2 

4N2 


«  -(WT  -  1) 


8N2 


(D.  25) 


Gathering  all  together 

ln7r(zm)  «  -  |  d  -  In  ^2*  d  (D.  26) 

To  summarise  the  approximation  technique;  the  Pearson  ratio,  the 
derivative  of  the  logarithm  of  the  ROC  character,  under  the  assumption 
that  the  range  of  the  variable  was  substantial,  suggested  a  match  with 
a  normal  ROC  curve.  By  matching  value'?  for  the  derivation  of  the  log¬ 
arithm  of  the  ROC  character,  a  relation  between  what  would  have  to  be 
the  matching  normal  index  and  the  parameters  of  this  problem  was 
obtained  (Eq.  4.  38).  Since  a  metastatic  transformation  of  a  normal 
ROC  curve  would  also  give  this  same  matching  value,  it  was  necessary 
to  calculate  an  actual  value  on  the  height  of  the  ROC  character,  or 
correspondingly,  its  logarithm.  Using  the  standard  power  series  and 
approximation-piled-on -approximation  technique  common  in  applied 


00| 


mathematics,  such  a  calculation  of  the  magnitude  of  the  ROC  char¬ 
acter  at  its  .  .ode  value  was  obtained.  This  is  Eq.  D.  26.  This  maximum 
value  of  the  ROC  cha  racter  matched  that  for  the  normal.  Since  the 
mode  is  extremely  close  to  zero,  the  logarithm  of  the  ROC  character 
is  parabolic  in  the  neighborhood  of  zero  and  the  magnitude  of  the  ROC 
character  corresponds  to  the  width  parameter  in  the  parabola  in  exactly 
the  same  way  that  a  normal  character  behaves.  We  conclude  that  the 
Pearson  III  characters  are  essentially  normal  in  the  limiting  case 


studied. 


APPENDIX  E 


A  CONIC  ROC  COMPUTER  PROGRAM 
V  r . 

This  appendix  contains  t>v.  details  of  a  computer  program 
mentioned  in  Section  5.  4,  A  regular  conic  HOC  is  specified  by  its 
minimum  slope  (called  LLO),  maximum  slope  (called  LHI),  and  one 
super  chance  point  (X^,  Y^).  These  three  items  of  information  are 
the  input  information  to  the  program. 

A  simplified  computer  flow  diagram  is  shown  in  Fig.  E.  1. 
The  equations  for  each  computation  block  constitute  the  remainder  uf 
this  appendix. 

SETUP  Certain  quantities  are  used  repeatedly  a"d  do  not 

depend  on  the  input  data.  These  are  computed  first. 


Z(I)  =  .  1(1-100) 
L(I)  =  eZ(I) 


X(l)  =  10 


1  <  I  <  200 


-4 


v-3 


) 


X(2)  =  10  v  i  <  j  <  50 

X(J>3)  =  .02J-.05  {  1  ^  J  W 

H<J>~=lnra$y  ) 


i  r 


CHECK  The  point  (Xq,  Yq)  must  lie  between  the  chance  diagonal 
and  the  Luce  ROC.  The  program  is  written  for  the  ROC  curve  slope 
range  (.  00005,  22000).  If  any  of  the  following  inequalities  is  TRUE, 
a  transfer  to  ”GOOF5  occurs. 
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0-  0 

Yq>  (LHI)X0 

(1-Y0)  <  (LL0)(1-X0) 

LHI  >  22000 
LL0<  .00005 

CONSTS  The  constants  in  the  algebraic  equation  for  the  ROC  and 

several  auxiliary  constants  are  computed. 

ZL0  =  In  (LL0) 

ZHI  =  In  (LHI) 

IL0  =  (integer  part  of)  10(ZL0)  +  101 

whenever  this  is  less  than  1,  set  IL0  =  1 
IHI  =  (integer  part  of)  10(ZHI)  +  100 

whenever  this  is  greater  than  200,  set  IHI  :=  200 

Cj  --  (LHI  +  LLO  -  2)/(l  -  LLO) 

C2  =  (1  -  (LL0)(LHI))/(1  -  LLO) 

C3='2(Y0-X0>'2 

A  =  C3(ClX0Y0  ♦  C2X“0  +  Y0  -  (LHI)X0) 

B  =  Q  -  A 
C  = 2Cg  +  A 
D  =  1 

E  =  -(LHI) 

A  =  B2  -  AC 

K  =  VAE2  -  2BDE  +  CD2 
C5  =  2(B  +  A(LHI)) 

LINLIN  The  ROC  curve  appropriate  for  plotting  on  linear-linear 

graph  paper  is  computed,  for  the  x  points  precomputed  in  SETUP.  The 
slope  of  the  ROC  curve,  the  likelihood  ratio,  is  also  computed. 

X  =  X(J) 

Y  =  Wax2  4  C5X  +  1  -  BX  -  1)/A  ,  .  J  ,  5, 

Y(J)  =  Y  ~ 

L  =  (-E  -  AC  -  BY)/ (D  +  BX  »■  AY) 


The  horizontal  coordinate  X  is  called  from  memory  to  compute  the 
vertical  coordinate  Y  and  the  slope  L„  X,  Y  and  L  are  then  printed 
out  and  Y  is  stored  for  later  use.  L  is  not  stored,  nor  does  it  corres¬ 
pond  to  the  stored  list  L(I). 

PIOFZ  >  This  computes  the  ROC  character.  The  general  equation 

i 

for  the  conic  ROC  character  is 

ff(z)  =  k(AeZ  +  2B  +  Ce  ‘J)  ^ 

The  computational  equations  are 

T  =  AL  +  2B  +  C/L 
CHAR  =  k/  (T  -/t) 

for 

L  =  LLO;  L(I)  ILO  <  I  <  IHI;  LHI 

After  each  character  value  is  computed,  L,  Z,  and  the  character  value 
are  printed. 

LOR  LOR  The  log-odds-ratio  coordinate  for  any  probability  p  is 

In  (p,/(l  -  p)).  This  was  computed  for  the  x  coordinate  in  SETUP.  The 
y  values  were  calculated  in  LINLIN. 

H  =  H(J) 

MPRJME  =  V  -  H 

The  program  terminates  (on  the  University  of  Michigan  machine)  when 
the  READ  DATA  statement  fails  to  encounter  new  data. 


J  <  52 


APPENDIX  F 


BINORMAL  COMPUTER  PROGRAMS 

This  appendix  contains  two  programs  for  finding  regular 
ROC  curves  related  to  an  irregular  binormal  ROC  curve.  Both  assume 
that  the  slope,  S,  is  betwt  n  zero  and  one  and  that  the  quality  Q,  (the  d* 
value  at  Y  =  .  50)  is  positive. 

The  first  program  searches  for  the  point  of  tangency  of 
the  ROC  curve  and  the  tangent  inferior.  It  then  performs  ” external 
rectification”  by  adjoining  the  tangent  inferior  to  the  regular  part  of 
the  ROC  curve.  The  printout  for  the  regular  part  of  the  ROC  curve 
consists  of  the  ROC  linear  coordinates,  the  slope  L,  its  natural  logarithm 
z,  the  slope  of  the  secants  from  (0,  0)  and  (1,  1)  to  the  ROC  point,  and 
the  ROC  character. 

The  second  program  operates  on  the  decision  axis  inferred 
by  the  original  ROC.  The  likelihood  ratio  decision  is  essentially  a  two- 
tail  test  whereas  the  original  ROC  is  a  one-tail  test.  The  program 
computes  the  original  ROC,  the  lover  tail  contributions,  the  total 
two-tail  ROC  curve,  and  the  ROC  character. 

Figure  F.  1  is  the  computei  flow  diagram  for  the 
external  rectification  program.  The  details  of  the  printed  comments 
and  equations  are  as  follows; 
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START1 


SETUP 


The  program  title  reads: 


REGULARIZED  BINORMAL  ROC  PROGRAM  NUMBER  ONE 
BINORMAL  ROCS  PLOT  AS  STRAIGHT  LINES  WITH 
SLOPE  LESS  THAN  ONE  ON  NORMAL -NORMAL  PAPER. 

IF  NOT  REGULARIZED  THEY  WOULD  STOP  SHORT  OF 
(1,  1)  AT  Z  =  ZQ.  TO  BE  REGULARIZED  THEY  ARE 
CONTINUED  ALONG  THE  TANGENT  INFERIOR  TO  (1,  1). 
THE  INPUT  DATA  ARE  IN  TERMS  OF  THE  ROC  ON 


NORMAL-NORMAL  PAPER.  Q  IS  THE  VALUE  OF 


DPRIME  AT  .  50  DETECTION,  AND  S  IS  THE  SLOPE. 


Tn(I)  =  .1(1  -  25) 

X(I)=  *(tn(D) 


1  <  I  <  63 


This  computes  the  normal  argument  from  -2.  4  to  +3.  8,  and  the  assoc 
iated  false  alarm  probability  range  is  .  0082  to  .  999928.  These  points 
are  uniformly  spaced  on  normal-normal  paper.  The  University  of 
Michigan  IBM  7090  MAD  Language  does  not  have  a  $  (  )  subroutine. 
The  actual  equation  used  is 

$(t)  =  .  5  +  .  5*ERF.  (V7T  t) 


ENDPT  The  EPT  comment  reads: 

THE  UNREGULARIZED  BINORMAL  ROC  WITH  SLOPE 
S  AND  .  50  DETECTION  DPRIME  Q  WOULD  RUN  FROM 
(0, 0)  TO  THE  POINT  OF  MINIMUM  SLOPE. 

Some  constants,  and  the  coordinates  of  the  point  of  minimum  slope  are 


then  calculated. 
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Whenever  S  >  1,  S  =  l/S 
A  =  SQ/(1  -  S2) 

B  =  (1  -  S2  )/2S2 

S2=  1/S1 

Zn  =  In  S  -  QSA/  2 

L  -  eZ° 

Lq  -  e 

XQ=  $(SA) 

Yq  =  $(A) 


REGROC  The  slope  of  the  ROC  curve,  L,  will  usually  fall  below  the 
secant  inferior  before  x  =  .  999928.  This  program  assumes  that  0.  1  steps 
in  normal  argument  is  sufficiently  fine  quantization  that  the  tangent  inferior 
corresponds  to  the  secant  inferior  which  first  falls  above  L.  ROC  curve 
points  are  computed  until  this  occurs,  or  until  I  =  63. 

TN  ■  tn(i> 
tsn  ■  s(tn  +  Q) 

C  =  .  39894228  EXP.  (-(T  2  +  T2  )/ 4) 

pi  =  c/(s1tsn-s2  y 

X  =  X(I) 

Y  =  «(Tsk) 

2  =  V  B(A  -  Ts/ 

L  =  ez 

SECSUP  =  Y/X 

SECINF  =  (1  -  Y)/  (1  -  X) 

If  the  slope  of  the  ROC  curve  is  still  above  the  secant  inferior  by 
x  = .  989928*  then  the  computation  terminates  and  the  following  OUT 
comment  is  printed: 

IN  THE  ABOVE  COMPUTATION,  NO  REGULARIZATION  WAS 
USED. 
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TANINF  The  TAN  comments  read: 

BINORMAL  ROC  IS  IRREGULAR  FROM  HERE  ON. 
TANGENT  INFERIOR  PORTION  FOLLOWS. 

The  loop  index,  J,  for  tnis  computation  is  initiated  at  the  last  used 

value  for  the  REGROC  loop  index,  I.  This  repeats  this  computation 

for  that  one  point,  and  then  continues  until  x  =  .  999928. 

X  =  X(J) 

Y  =  1  -  T  +  TX 

SECSUP  =  Y/X 

The  program  terminates  (on  the  University  of  Michigan  machine) 
when  the  READ  DATA  statement  fails  to  encounter  new  data. 

The  flow  diagram  for  the  internal  rectification  program 
is  given  in  Fig.  F.  2. 

% 

FRONT  The  front  page  of  each  program  bears  the  following 

program  title: 

REGULARIZED  BINORMAL  ROC  PROGRAM  NUMBER  TWO. 
BINORMAL  ROCS  PLOT  AS  STRAIGHT  LINES  WITH 
SLOPE  LESS  THAN  ONE  ON  NORMAL-NORMAL  PAPER. 

IF  NOT  REGULARIZED  THEY  HAVE  A  POINT  OF 
MINIMUM  SLOPE  SHOBT  OF  (1, 1).  TO  BE  REGULARIZED 
A  NEW  DECISION  AXIS  BASED  ON  LIKELIHOOD  RATIO 
IS  USED.  THE  INPUT  DATA  ARE  IN  TERMS  OF  THE 
ROC  ON  NORMAL-NORMAL  PAPER.  Q  IS  THE  VALUE 
OF  DPRIME  AT  .  50  DETECTION,  AND  S  IS  THE  SLOPE. 
TANGENT  INFERIOR  P(A/SN)  IS  GIVEN  VS  P(A/N) 

OF  THE  FINAL  ROC. 


diagram  for  internal  rectification 
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START 


In  addition  to  the  data  Q  and  S,  the  input  data  contains 


an  integer,  No. ,  and  a  word,  type.  To  obtain  one  curve,  set  No.  =  1. 
To  obtain  N  parallel  curves,  set  No.  =  N,  and  type  =  SAME.  To 
obtain  N  ROC  curves  that  have  a  common  chance  diagonal  point  set  No. 
=  N,  type  =*  FAN.  (Such  a  "fan"  is  a  family  of  ROC  curves  modeled  as 
a  zero-mean  unit-variance  noise,  with  SN  distribution  normal  with 
mean  Q  and  standard  deviation,  1  +  K  a  number  of  curves  are 

calculated,  Q  will  be  used  both  as  the  initial  value  and  as  the  incre¬ 
mental  Q  value. 


TYPING 


The  SAME  comment  reads:  "I  SHALL  NOW  COMPUTE 


A  NUMBER  OF  CURVES  WITH  IDENTICAL  NORMAL-NORMAL  SLOPE. " 
It  is  followed  by  the  values  of  S,  A  t  No. 

If  the  type  is  FAN 
Tpt  =  sQ/(1-S) 

pt  pt' 

kT.0  =  !/  T  . 


The  FAN  comment  reads:  ”1  SHALL  NOW  COMPUTE  A  FAN  OF 
CURVES  THROUGH  THE  COMMON  CHANCE  DIAGONAL  POINT. "  It 
is  followed  by  the  values  of  X^,  A,  No, ,  kJAS  ' 


NLOOP 


The  equations  in  the  ’’compute  constants"  block  are 
B  *  1  -  S* 

Zft  =  In  S  -  S*  Q* /2B 

i  *0 

Lo  =  e 

DPDET  «  Q 
DPSUBE  *  2SQ/(1  +  Si 
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DPFA  =  QS 

t0  =  s8q/b 
C  =  2T0 

— NI<jER  The  Pa^e  heading  reads:  ’’THE  SLOPE  ON  NORMAL- 

NORMAL  PAPER,  ANli  THE  DPR1ME  VALUES  OF  THE  UNREGULARIZED 
HOC  AT  .  50  DETECTION,  NEGATIVE  DIAGONAL,  AND  .  50  FALSE 

ALARM  ARE  .  This  is  followed  by  the  values  of  S,  DPDET, 

DPSUBE,  DPFA,  then  followed  by  Z  and  L  . 

0  0 


MAIN 


The  MAIN  equations  are 

v-vc 

T3  =  S(T1  -  Q) 

T4  =  S(T  -  Q) 

VVT1 

XORIG  =  ^(-Tp 

yorig  =  <o(-t3) 

XADD  =  $  (T„) 

YADD-  3>(Tp 

X  ^  XORIG  =  XADD 
Y  =  YORIG  +  YADD 

Z  =  ZQ  +  BT^  5Ta 

PI  =  .  39894228  e-’  5  e 


5T*, 


+  e 


YTAN  =  1 


/  BT. 


L0  +  L0X 


These  programs  were  used  to  obtain  regularized 
binormal  examples  for  Chapter  VUI. 


APPENDIX  G 


MANIPULATION  OF  BINOMIAL  SUMS 


Remark  1 


X 


=  2;  cfd-xiV’i  =4>X  =(-l)k(n-k)Cn  l  (-1) 

n»K  j=0  1  -  n>K  Km=0 


m 


n-  m 


CkXn 

m 


-m 


(G.1) 


Proof: 


l  c,"  (1  -  X)jx"*j  =  I  cn  x""'  i 

j=0  J  j*0  J  m=0 


=  I  E  (-l)*-"1 

0<  m<  j<  k 


cn  CJ 

3  m 


yll-m 


Now 


pnpj  nl _ jj _ n!  _ l _ 
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Let  r  =  j  -  m  replace  j 

0  <  m  <  j  <  k  =  0  <  m  <  k  and  0  <  r  <  k-m 
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But  Eq.  24. 1.  4  of  Ref.  6  is 
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Remark  3 
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Proof ; 
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For  m  =  0  this  term  is  zero;  the  sum  now  runs  from 
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”■  *,*,"*CT  The  first  problem  in  the  theory  of  signal  detectability  deals  with  the  decision] 
between  two  alternative  responses^  corresponding  to  two  possible  classes  of  causes 
of  an  observation.  When  the  goal  pf  a  decision  process  is  to  achieve  the  highest 
quality  of  terminal  decision,  the  Receiver  Operating  Characteristic  curve  (ROC  curve 
contains  all  of  the  information  necessary  for  the  evaluation  of  the  decision  process. 
This  present  work  introduces  the  l$OC  character,  which  is  isomorphic  to  the  ROC 
curve.  THe  formal  development  is  based  on  two  key  facts.  The  first  is  the  fundamen 
tal  theorem:  if  £(X)  is  the  likelihood  ratio  of  an  observation,  then  the  likelihood 
ratio  of  f  1  is  £  1  itself;.  The  second  is  the  main  theorem  on  ROC  characters:  each 
ROC  character  ii  isomorphic  to  a  univariate  probability  distribution  that  possesses  a 
moment  generating  function.  The  Character  convolution  theorem  and  the  character 
addition  th^orenn follow  directly  frpm  these.  Families  of  ROC  curves  are  developed 
from  the  main  theorem  on  ROC  characters.  The  normal,  binormal,  Q- table,  power, 
and  several  discrete  families  of  ROC  curve  have  appeared  in  the  literature .  The  new 
families  include  the  Pearson  jtype  ill,  Fisher- Tippett  doubly  exponential,  H-type, 
Poisson,  and  the  regular  conics.  Additional  families  are  generated  from  these  by 
use  of  the  metastatic  relation,  and  the  convolution  and  addition  theorems.  ROC 
curves  contain  information  about  the  performance  of  other  two- cause  decisions  be¬ 
sides!  the  t^o-r espouse  decision.  Several  are  considered  that  are  used  in  the  testing 
of  human  perception;  namely,)  the  symmetric  forced  choice  decision,  type  n  decision^ 
am - -  tyocedfoft  pjad  the^alvBis  nt  a  deelalnn  hasp  A  on  the  reports  of  If 
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